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DEFINITIONS 

IDA  publlshu  IlM  tollowlng  to  ro|Mrt  Sio  ranits  ol  Hs  «nit. 

Reports 

Reports  ore  the  ntost  ootkortloliM  onO  mosi  eonfilly  eonsMorod  pndecit  IDA  ptblMtst. 
They  normolly  omkody  rosolts  ol  aio|or  pro|oeis  wbleh  (a)  have  a  dinct  baartop  on 
decisions  afloctini  maior  profrann,  (b|  addrass  Issaos  ol  slinnieant  concern  to  On 
Execotlve  Brancb,  Ibo  Conirau  and/or  Ibo  pnblic,  or  (c)  addrass  Issaos  that  bairo 
signiflcant  oconomic  ImpllcaUons.  IDA  Roporis  am  roslastad  by  oatsida  pantls  ol  oiparis 
to  ensam  thalr  high  qnallly  and  ralennca  to  the  prebloms  stndlad,  and  they  am  mlaasod 
by  tba  Pmsidoal  ol  IDA. 

Group  Reports 

Granp  Raperts  record  the  lindings  and  msollte  ol  IDA  ostabllsbod  working  granps  and 
panels  composed  ol  saniar  hidMduals  addressing  major  Issaos  which  olherwisa  weald  bo 
the  sobjoel  at  an  IDA  Roport.  IDA  Group  Reports  am  msiswsd  by  tba  senior  Indhridoals 
rosponsiblo  lor  the  projacl  and  others  as  soloclad  by  IDA  to  ensam  thalr  high  qaallty  and 
ralesance  to  the  prebloms  shidM,  and  am  rsloasad  ^  tba  Pmsidsnt  ol  IDA. 


Papers 

Papers,  also  antborltativo  and  caralnlly  considomd  prodnets  ol  IDA,  addma  stadias  that 
am  narrower  in  scope  than  those  covered  in  Reports.  IDA  Papers  am  mvlowad  la  ensam 
that  they  meet  the  high  standards  axpectod  ol  rolemed  papem  In  prolsssional  Jeunmls  or 
tormai  Agency  raperts. 


Documents 

IDA  Docamonts  am  used  lor  the  convenience  at  the  spoosom  or  the  analysts  (a)  to  mcoid 
substantive  work  done  In  quick  raacllen  studios,  (b)  to  record  the  precoodings  ol 
conlersncos  and  mootings,  (c)  to  make  avallaWo  pmllminary  and  tantatlva  mnlts  ol 
anatysos,  (d)  to  record  data  developed  in  Uio  coorsa  el  an  InvastlgaUon,  or  (e)  to  torward 
inlormation  that  is  essentially  nnanalyzsd  and  unevsluated.  Tho  raviow  ol  IDA  Docamonts 
Is  suited  to  their  content  and  Intended  use. 


The  work  reported  In  ttils  docomenl  was  caoductod  under  contract  MDA  903  S9  C  0003  lor 
ttie  Department  ol  Delensa.  The  publication  el  Oils  IDA  decament  does  not  Indicate 
endorsamont  by  Oia  Department  ol  Dolonsa,  nor  should  Uio  contents  bs  cansiraod  as 
raflocUng  Oia  official  posilion  ol  Oiat  Agency. 
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PREFACE 


The  work  reported  in  this  document  was  performed  for  the  OfHce  of  Industrial 
Engineering  and  Quality  within  the  Production  Resources  Office  in  the  Office  of  the 
Assistant  Secretary  of  Defense  for  Economic  Security  under  the  technical  cognizance  of  the 
Armament  Research,  Development,  and  Engineering  Center  at  Picatinny  Arsenal,  NJ, 
Quality  Assurance  Directorate  under  Contract  MDA  903-89C-0003.  The  objective  of  the 
task,  Government-Industry  Standardization  of  Product  Acceptance  Based  on  Process  Data, 
was  to  help  devise  a  new  Department  of  Defense  (DoD)  approach  to  quality  assurance 
practices,  including  the  development  of  a  standard  acceptable  to  both  DoD  and  industry  to 
move  DoD  away  from  accepting  product  by  end-item  inspection  to  accepting  product  based 
on  the  contractor’s  quality  system  and  use  of  process  controls. 

Reviewers  of  this  report  were  Mr.  Chris  Jehn  and  Dr.  David  Graham  of  the 
Strategy,  Forces  and  Resources  Division  of  the  Institute  for  Defense  Analyses  (IDA),  and 
Dr.  Donald  Ermer,  Professor  of  Mechanical  and  Industrial  Engineering  at  the  University  of 
Wisconsin-Madison  and  holder  of  the  first  Proctor  and  Gamble  Bascom  Professorship  in 
Total  Quality. 
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EXECUTIVE  SUMMARY 


Under  increasing  pressure  to  change  its  way  of  doing  business  and  adopt 
commercial  products,  practices,  specifications,  and  standards,  DoD  approached  the 
Institute  for  Defense  Analyses  (IDA)  to  conduct  a  study  that  would  ultimately  lead  to  new 
quality  assurance  practices  for  Defense.  IDA  was  approached  because  of  its  vast  previous 
experience  with  concurrent  engineering  and  quality  management  practices  within  industry 
and  the  Government.  The  subject  was  controversial  because  of  high  Congressional 
visibility  and  a  public  perception,  based  on  sampling  inspection  standards  still  in  place,  that 
DoD  was  willing  to  accept  defective  product.  Objectivity  and  independence  were  required 
to  develop  a  new  standard  practice  that  both  DoD  and  industry  would  be  comfortable  with. 

IDA  set  out  to  analyze  the  best  commercial  practices,  specifications,  and  standards 
and  prepare  a  draft  of  a  new  standard  that  provided  a  replacement  for  the  current  sampling 
inspection  standards,  required  manufacturing  process  and  statistical  process  controls  for  the 
most  critical  characteristics  of  the  product,  and  provided  an  incentive  to  all  producers  to  use 
effective  quality  practices.  IDA  enlisted  the  help  of  recognized  acceptance  sampling  and 
quality  assurance  experts.  Dr.  Edward  G.  Schilling,  a  professor  at  the  Rochester  Institute 
of  Technology  and  author  of  numerous  books  on  acceptance  sampling,  and  Mr.  Seymour 
J.  Lorber,  retired  Deputy  Chief  of  Concurrent  Engineering  for  Army  Materiel  Command 
(AMC),  were  hired  as  consultants.  IDA  also  worked  with  a  recognized  industry 
association  Statistical  Process  Control  (SPC)  committee  with  industry,  academia,  and  DoD 
participants  to  develop  the  standard  and  get  it  accepted. 

In  addition  to  developing  the  standard,  IDA  reconunended  changes  to  DcD  quality 
assurance  practices  in  general  throughout  the  course  of  the  study.  In  particular,  IDA 
recommended  that  DoD  not  only  authorize  use  of  the  International  Organization  of 
Sandards,  ISO  9001,  Quality  Systems — Model  for  Quality  Assurance  in  Design, 
Development.  Production,  Installation,  and  Servicing,  but  aggressively  plan  for  its  full 
adq)tion  over  a  reasonable  period  of  time.  With  this  authorization,  IDA  recommended  that 
DoD  not  endorse  or  require  certification  by  a  third  independent  party.  This  restriction 
would  not  prevent  the  contractor  from  using  consultants  or  other  services  to  implement  a 
quality  program.  With  regard  to  this  recommendation,  it  is  noted  that  OSD,  on 
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14  February  1994,  authorized  the  use  of  the  ISO  9000  and  its  American  National  Standards 
Institute/American  Society  of  Quality  Control,  ANSI/ASQC  Q90,  series  of  quality 
standards  for  new  programs  and  as  appropriate  for  ongoing  programs.^ 

A.  RECOMMENDATIONS  TO  IMPROVE  DEFENSE  QUALITY 
ASSURANCE  ACTIVITIES 

Efforts  by  DoD  during  the  past  several  years  to  improve  its  acquisition  quality 
assurance  operation  have  resulted  in  many  significant  improvements.  In  the  interest  of 
further  improvement,  IDA  recommends  that  DoD  continue  to  increase  procurement  of 
commercial  products,  reduce  government  oversight  of  contractors,  utilize  commercial 
specifications  and  standards  wherever  possible,  strive  for  the  prevention  of  defects  by  the 
implementation  of  integrated  product  and  process  development  (IPPD),  and  accommodate 
dual-use,  commercial/military  integration  in  production  lines.  Specifically,  IDA 
recommends  the  following: 

•  Replace  MIL-STD-105,  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  with  its  commercial  equivalent,  ANSI/ASQC  Zl.4-1993. 

•  Continue  to  use  ANSI/ASQC  Z1.4  plans  for  products  that  are  of  relatively 
simple  design,  described  by  standard  Technical  Data,  and  for  which  experience 
has  demonstrated  no  problems. 

•  Replace  MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection  by 
Variables  for  Percent  D^ective,  with  its  commercial  equivalent,  ANSI/ASQC 
Zl.9-1993. 

•  Request  ASQC  to  develop  an  ANSI/ASQC  standard  to  replace  MIL-STD- 1235 
and  cancel  MIL-STD- 1235. 

•  Replace  MIL-I-45208,  Inspection  System  Requirements  with  the  new  draft 
standard,  DoD  Preferred  Methods  for  Acceptance  of  Product  (Appendix  D  to 
this  paper). 

B .  RELATED  ACTIVITIES  IN  DOD 

While  this  effort  was  initiated  in  1991,  it  has  dovetailed  with  recommendations  in 
two  recent  activities.  The  joint  Military  and  National  Aeronautical  and  Space 
Administration  (NASA)  Handbook,  Interim  Guidance  on  the  Application  of  ISO 


^  John  Deutch,  Under  Secretary  of  Defense.  Memorandum  for  Secretaries  of  the  Military  Departments, 
Directors  of  Defense  Agencies,  Use  of  Commercial  Standards  in  the  Department  of  Defense  (DoD), 
14  February  1994. 
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9000! ASQC  Q90  Series  Quality  Systems  in  Standards,  was  issued  on  4  February  1994. 
Its  purpose  is  “to  assist  contracting  activities  that  have  decided  to  use  Q91  or  Q92  quality 
system  standards  with  domestic  contracts  and  seek  guidance  for  doing  so.” 

In  April  1994,  the  Report  of  the  Process  Action  Team  on  Military  Specifications 
and  Standards  was  also  issued.  It  makes  recommendations  for  a  process  focus,  AQL 
elimination,  and  reduced  contractor  inspection  and  test.  Specifically,  under  Oversight,  the 
report  recommends  two  specific  tasks  related  to  the  draft  standard  and  the  recommendations 
in  this  paper 

•  “Deputy  Secretary  of  Defease  issues  a  policy  memorandum  emphasizing 
greater  use  of  process  controls  in  lieu  of  development  and  production  testing 
and  inspection.”^ 

•  “Develop  a  priority  action  list  of  military  specifications  containing  fixed 
allowable  defect  level  measures  such  as  acceptable  quality  levels  or  lot 
tolerance  percent  defect.  Initiate  action  to  eliminate  requirements  for  these 
defect  measures.”^ 

And  under  Contractor  Test  and  Inspection,  the  report  states:  “The  contractor  shall  certify  to 
the  government  that  the  item  or  items  offered  for  acceptance  and  delivery  satisfy  the 
requirements  of  the  specifications  through  process  controls  and  inspections.”^ 

The  draft  standard  in  Appendix  D  fits  with  these  two  recent  activities  and  should  be 
approved  as  a  military  or  commercial  standard  with  the  recommended  elimination  of  the 
other  documents  listed  in  Section  A. 

C.  SUMMARY  OF  RECOMMENDATIONS 

1.  Sampling  Inspection  Requirements 

•  Cancel  MIL-STD-105,  MIL-STD-414  and  MIL-STD- 1235,  Single-  and  Multi- 
Level  Continuous  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  Functional  Curves  of  the  Continuous  Sampling  Plans. 

•  Use  ANSI  standards  equivalent  with  canceled  standards. 

•  Urge  ASQC  to  prepare  an  ANSI  standard  to  replace  MIL-STD- 1 235. 


^  Report  of  the  Process  Action  Team  on  Military  Specfications  and  Standards,  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology.  April  1994,  p.  100. 

3  Ibid. 

*  Ibid.  p.  107. 
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2  .  Quality  Management  Standards 

•  Implement  aggressive  efforts  to  replace  MIL-Q-9858  with  ANS I/ASQC  Q9 1 . 

•  Replace  MIL-I-45208  with  a  new  standard,  DoD  Preferred  Methods  for 
Acceptance  of  Product  (Appendix  D  to  this  report). 

D.  “DEFECT  ANTICIPATION”  FOLLOW-ON  EFFORT 

This  study  had  as  its  focus  the  encouragement  of  improvements  in  defect  prevention 
and  process  control  as  a  means  for  improved  quality  in  DoD.  To  make  truly  significant 
quality  (both  performance  and  cost  effectiveness)  gains,  the  challenge  is  for  DoD  to 
develop  d^ect  anticipation  practices.  These  are  efforts  that  influence  the  product  designs  to 
improve  their  robustness  and  tolerance  to  the  vagaries  of  the  production  floor  as  well  as  the 
battlefield  environment.  When  defect  anticipation  and  variability  reduction  come  together, 
the  prospects  of  Process  Based  Acceptance  may  become  real. 
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I.  CHANGING  INDUSTRIAL  ENVIRONMENT 


The  new  national  security  environment  embodies  elements  that  previously  were 
absent.  It  now  includes  national  economic  security  that  functions  within  an  integrated, 
commercialAnilitary  national  industrial  base.  This  new  mission  for  the  Department  of 
Defense  (DoD)  and  the  austere  budget  constraints  it  faces  lead  to  the  need  for  high  quality 
products  at  an  affordable  cost  utilizing  best  commercial  products,  facilities,  and  practices, 
including  commercial  specifications  and  standards.  This  paper  addresses  the  Quality 
Assurance  function  of  DoD  in  this  new  environment,  and  this  first  chapter  discusses  the 
changing  commercial  industrial  environment  with  which  DoD  practices  will  be  contrasted. 

A.  PRIOR  TO  1980s:  QUALITY  BY  INSPECTION 

Products  that  are  mass  produced  are  manufactured  by  “lots."  A  lot  is  a  speciried 
homogeneous  collection  of  production  runs  or  shifts  on  a  single  line.  Inspection  is  the 
process  by  which  characteristics  of  the  product  coming  off  the  line  (end  items)  are 
inspected  for  conformance  to  the  product’s  specification.  Inspection  can  encompass  100 
percent  of  the  items  or  a  random  sample  of  the  lot — a  less  costly  process  incurring  some 
calculated  risk.  The  latter  process,  called  sampling  inspection,  is  based  on  standards  that 
are  indexed  by  the  amount  of  tolerable  risk  as  dictated  by  Acceptable  Quality  Levels  (AQLs) 
and  Lot  Tolerance  Percent  Defectives  (LTPDs).  Sampling  plans  include  the  lot  size  with 
related  sample  size  and  the  acceptAeject  criteria.  End  items  caii  be  inspected  by  attributes, 
for  which  either  a  go  or  no  go  decision  is  made  often  by  simple  gages  (e.g.,  the  outer 
diameter  of  a  bolt),  or  by  variables,  for  which  a  precise  measurement  is  taken  of  the 
characteristic  that  has  to  be  within  a  certain  tolerance  of  the  product  specification  (e.g., 
thread  width). 

Prior  to  World  War  11,  during  the  1920s  and  1930s,  the  general  industry  practice 
was  100  percent  inspection  of  a  lot  or  of  some  sample  of  the  lot  (e.g.,  10  percent). 
Western  Electric  introduced  the  Dodge-Romig  inspection  sampling  plans  for  internal  use  in 
the  1920s.  These  plans  provided  single  or  double  sampling  tables  categorized  to  achieve 
(1)  minimum  inspection  labor  or  (2)  a  limit  on  the  amount  of  defective  product 
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After  World  War  II,  industry  applied  inspection  sampling  extensively  as  a  result  of 
war  time  experience  and  the  effons  of  the  new  American  Society  for  Quality  Control 
(ASQC).  The  Dodge-Romig  attributes  sampling  tables  for  lot-by-lot  inspection  were  made 
commercially  available  in  1959.'  Many  companies  used  MIL-STD-105,  Sampling 
Procedures  and  Tables  for  Inspection  by  Attributes,  which  was  first  published  as  MIL- 
STD-105A  in  1950.  The  commercial  equivalent  to  MIL-STD-105D  (1963)  was  issued  as 
American  National  Standards  Institute/American  Society  for  Quality  Control  (ANSI/AS(^) 
Z1.4  in  1981.  MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection  By 
Variables  for  Percent  Defective,  was  first  published  in  1957,  with  its  commercial 
equivalent,  ANSI/ASQC  Z1 .9,  in  1980.  Many  companies  devised  their  own  versions  of 
the  published  plans.  The  aircraft  industry  employed  100  percent  inspection  because  of  its 
stringent  safety  requirements. 

B.  mid-1980s:  U.S.  DISCOVERS  DEMING 

During  the  mid  1980s,  U.S.  industry  was  in  an  upheaval.  U.S.  producers  who 
complacently  held  the  market  share  for  so  long  were  now  starting  to  lose  market  share  to 
the  Japanese.  U.S.  manufacturers  could  no  longer  afford  the  expensive  scrap  and  rework 
necessary  to  achieve  a  certain  level  of  quality  when  Japanese  msi.wfacturers  were 
eliminating  scrap  and  rework  and  achieving  a  higher  level  of  quality.  Television  shows 
engrossed  viewen  with  titles  like  Japan  Can,  Why  Not  America?  This  crisis  forced  senior 
management  to  embrace  the  “Deming”  approach  and  become  fully  aware  of  the  Total 
Quality  Management  (TQM)  philosophy,  including  improved  customer  and  supplier 
relationships  and  a  focus  on  continuous  improvement  and  control  of  the  processes  through 
statistical  process  control  (SPC).  The  leaders  in  the  quality  movement,  Philip  B.  Crosby 
and  Dr.  W.  Edwards  Deming,^  were  in  great  demand  as  speakers  both  for  television 
shows  and  in  workshops  and  seminars  across  the  nation.  Many  companies  devised  their 
own  quality  programs,  such  as  Motorola’s  6  Sigma  approach,  Ft;rd’s  (^ality  First  (QIOl), 
and  Boeing’s  Dl-9000.  duality  management  and  process  controls — a  focus  on  prevention 
and  improvement — were  being  recognized  worldwide  for  the  competitive  advantage  they 
offered.  Instead  of  “inspecting  quality  in’’  after  production,  resulting  in  rework  and  scrap 


'  HP.  Dodge,  and  H.G.  Romig,  Sampling  Inspection  Tables,  2nd.  ed.,  John  Wiley  and  Sons,  New 
York.  1959. 

2  Philip  B.  Crosby,  Quality  is  Free,  1979;  W.  Edwards  Denting,  Out  of  the  Crisis,  MIT,  1982. 
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and  the  additional  costs  they  entail,  process  controls  aim  to  minimize  scrap  and  rework  by 
the  prevention  of  defects. 

During  the  latter  part  of  the  1980s,  the  International  Organization  of  Standards 
(ISO),  ANSI,  and  the  ASQC  were  in  final  coordination  with  the  ISO  9000  series  of  Quality 
System  Standards  and  Guidelines,  as  follows: 

•  9(X)0:  Quality  Management  and  Quality  Assurance  Standards — Guidelines  for 

Seleaion  and  Use 

•  9001:  Quality  Systems — Model  for  Quality  Assurance  in  Design, 

Development,  Production,  Installation  and  Servicing 

•  9002:  Quality  Systems — Model  for  Quality  Assurance  in  Production  and 

Installation 

•  9003:  Quality  Systems—Model  for  Quality  Assurance  in  Final  Inspection  and 

Test 

•  9004:  Quality  Management  and  Quality  System  Elements — Guidelines 

The  ANSI/AS(3C  Q90  series  of  documents  ((390,  Q91,  Q92,  <393,  (394)  are  the 
U.S.  equivalents  of  the  ISO  series.  Commercial  application  of  the  ISO  9000  series  began 
expanding  in  the  United  States  and  the  world.  Industry  was  voluntarily  changing  its 
approach  to  quality  by  using  ISO  9001  or  ISO  9002  as  the  model  for  operations.  The 
European  Community  (EC)  announced  it  would  require  an  ISO  9000  series  Quality  System 
in  any  company  it  did  business  with.  Industry  recognized  that  prevention  was  a  means  to 
control  cost  as  well  as  to  meet  the  requirements  of  major  customers. 

C.  1990s:  HIGH  QUALITY  AT  LOWER  COST  BEING  ACHIEVED 
THROUGH  NEW  MANAGEMENT  PRACTICES 

Into  the  1990s  commercial  industry  continued  its  trend  toward  using  TQM 
principles,  and  the  defense  industry  followed.  In  1992,  the  prestigious  Malcolm  Baldrige 
National  <3uality  Award  went  to  the  Defense  Systems  and  Electronics  Group  at  Texas 
Instruments,  Incorporated.  DoD  itself  began  pursuing  such  advanced  quality  concepts  as 
quality  in  source  selection;  specification  streamlining;  leadership  and  management 
commitment;  employee  participation;  quality  improvement  training  and  development; 
quality  performance  measurement  and  recognition;  prevention-based  quality,  variability 
reduction,  key  supplier  involvement;  line-proofing;  identification  of  key  process 
characteristics;  and  manufacturing  risk  management. 
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DoD  was  beginning  to  see  that  the  use  of  best  commercial  practices  by  defense 
contractors  could  improve  quality  and  reduce  costs — but  not  without  many  changes  in  the 
way  DoD  does  business.  Customer  satisfaction  and  involvement  resulting  in  a  quality 
product  meeting  the  user’s  needs  at  lower  cost  requires  an  Integrated  Product  and  Process 
Development  (IPPD)  environment  with  close  prime/subcontractor  arrangements  and  a 
relationship  of  trust  between  the  contracttn^  and  DoD. 


1-4 


II.  PRESSURES  FOR  DOD  TO  CHANGE 


A.  HISTORY  OF  QUALITY  ASSURANCE  PRACTICES  IN  DOD 

Defense  acquisition  quality  assurance  (QA)  practices  consist  of  three  basic  activities; 
Sampling  Inspection  Standards,  Quality  Management  System  SpeciHcations,  and 
Government  Quality  Assurance  Oversight  of  Contractor  Operations.  The  sections  below 
describe  the  three  current  military  sampling  inspection  standards,  the  two  military 
specifications  for  a  quality  management  system,  and  the  DoD  efforts  to  change  the 
customer/supplier  relationship. 

1.  Sampling  Inspection  Standards 

Prior  to  World  War  H,  in  the  1920-1930  time  frame.  Government  inspectors 
inspected  100  percent  of  military  products  for  ctHiformance  with  specification  requirements 
and  returned  defective  product  to  the  producer  for  repair  or  replacement.  Sampling 
inspection  procedures  were  introduced  in  1942,  when  the  U.S.  Army  published  sampling 
tables  including  single  and  double  sampling  plans  indexed  by  Acceptable  Quality  Levels 
(AQLs).  These  tables — 

•  Considered  AQLs  to  be  the  desired  prcx«ss  average. 

•  Proteaed  the  producer  frcxnrejectitxi  of  lots  better  than  AQL. 

•  Imposed  more  stringent  requirements  when  quali^  history  was  poor. 

•  Imposed  more  stringent  criteria  for  serious  defects. 

•  Provided  economies  when  quality  history  was  good. 

•  Set  sample  size  requirements  based  on  lot  size. 

By  1945,  several  similar  plans  were  available.  For  example,  the  Navy  had  multiple 
sampling  plans  in  Tables  and  Procedures  by  the  Statistical  Research  Group.  In  1950, 
MIL-STD-105A,  Sampling  Procedures  and  Tables  for  Inspection  by  Attributes,  was 
published,  and  in  1963  it  became  an  ABCA  (America,  Britain,  Canada,  and  Australia) 
Standard.  The  initial  beneHts  to  the  DoD  of  this  approach  during  WW  n  included 
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improved  quality  of  product  received  and  a  substantial  reduction  in  the  number  of 
government  inspectors  required. 

a.  Military  Inspection  Engineering  Activities,  1940s 

The  introduction  of  sampling  during  WW II  required  Army  and  Navy  engineering 
activities  to  specify  inspection  standards  and  to  classify  the  AQLs.  Standard  Inspection 
Procedures  (SIPs)  were  developed  in  which  product  characteristics  were  defined  and 
classified  by  their  importance  (Critical,  Major,  Minor).  The  SIPs  provided  standards  fm* 
“good”  or  “bad,”  were  not  contractual,  and  were  issued  for  the  government  inspectors  to 
use.  Special  product  training  for  government  inspectors  included  sampling  practices  and 
use  of  inspection  equipment.  Inspection  equipment  was  issued  for  the  government 
inspectors  as  a  means  of  performing  government  inspection. 

b.  Sampling  Practices,  1950-1970 

Within  DoD,  MIL-STD-IOS^  and  supporting  sampling  standards,  such  as 
MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection  by  Variables  for  Percent 
Defective?-  and  MIL-STD-123S,  Single  and  Multi-Level  Continuous  San^ling  Procedures 
for  Inspection  by  Attributes,  Functional  Curves  of  the  Continuous  Sampling  Plans?  were 
used  extensively.  In  the  mid-1950s,  an  Instruction  was  issued  to  abolish  the  SIPs  and 
instead  include  the  classification  of  defects,  the  AQLs,  and  the  required  inspection 
equipment  in  Section  4  of  the  Item  Specification.  All  inspection  was  to  be  performed  by 
the  contractor  unless  reserved  for  the  exclusive  action  by  the  Government 

c.  Technical  Data  Package 

The  Product  Specification  serves  as  the  principal  element  of  the  Technical  Data 
Package  (TDP)  and  includes  a  Section  3,  Technical  Requirements,  and  Section  4,  Quality 
Assurance  Requirements.  Section  3  of  the  Product  Specification  contains  all  product 
technical  requirements  including  drawings  and  parts  list,  and  excluding  packaging,  which 
is  in  Sectitm  5.  Section  4  of  the  Produa  SpeciHcation  contains  all  the  test,  inspection,  and 
examination  requirements  and  the  specification  quality  system  requirements. 


1  Fust  published  as  MIL>STD-10SA  in  1950. 

2  Hist  published  in  1957. 

3  MIL-STD-1235A  published  in  1974. 


Since  the  1950s,  MIL-STD-105  has  served  as  the  basis  for  prescribing  required 
product  sampling  and  level  of  risk  to  be  tolerated  in  defense  Product  Specifications.  Based 
on  specification  language,  the  producer  is  required  at  a  minimum:  (1)  to  perform  the 
sampling  prescribed  in  the  contract;  (2)  to  take  appropriate  disposal  action  on  the  basis  of 
the  inspection  results  and  the  sampling  plan  accept/reject  criteria;  and  (3)  to  submit  the 
product  that  meets  requirements  together  with  the  supporting  inspection  results  to  the 
government  representative  for  acceptance  puiposes. 

2.  Quality  Management  System  Specifications 

Two  military  specifications,  Quality  Program  Requirements  and  Inspection  System 
Requirements,  currently  provide  a  two-tier  system  for  the  procuring  activities  to  specify 
quality  management  system  requirements. 

The  Air  Force  pioneered  requirements  for  contractor  quality  program  requirements 
during  the  early  1950s  when  it  issued  MIL-Q-5923  AF.  These  requirements  were  aimed  at 
major  contractors  engaged  in  the  development,  production,  and  selected  service  tasks  for 
aircraft  and  missile  systems.  Following  the  Air  Fwce  example,  the  other  Services  initiated 
similar  requirements  for  their  products  of  similar  complexity.  Finally,  the  OSD  staff 
brought  all  parties  together  in  the  late  1950s  to  establish  a  coordinated  approach:  Military 
Specirication  MIL-Q-9858,  Quality  Program  Requirements.  The  Air  Force  experience 
demonstrated  that  control  of  the  engineering  and  production  processes  at  the  facilities  of  the 
prime  contractors  and  subcontractors  was  necessary  to  assure  delivery  of  acceptable 
product  Hie  traditional  practice  of  end  item  inspection  simply  would  not  provide  sufficient 
evidence  of  a  quality  product 

As  the  requirement  for  a  quality  system  was  imposed  by  contract,  many  contractor 
senior  managers  in  the  defense  industry  believed  it  to  be  an  unnecessary  burden  or  an 
oveihead  cost  that  led  to  increased  costs  and,  therefore,  needed  to  be  applied  on  a  restricted 
basis.  Allowable  quality  assurance  costs  were  frequently  set  by  negotiation  between  the 
contractor  and  Government.  Independent  surveys  of  military  contractors  to  identify  cost 
drivers  or  unnecessary  requirements  always  included  Quality  Program  Requirements  high 
up  on  resulting  survey  reports.  The  extensive  use  of  100  percent  inspection  throughout  the 
manufacturing  operation  due  to  the  critical  safety  and  high  performance  requirements  for 
aircraft  and  missile  products  also  held  back  the  application  of  process  controls  and 
Statistical  Quality  Control  (SQC)  methods.  While  the  application  of  Quality  Program 
Requirements  may  not  have  been  efficient,  the  resulting  product  performed  as  expected. 
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The  other  Military  Specification,  MIL-I-45208,  Inspection  System  Requirements, 
originally  prepared  by  the  Army  in  the  late  19S0s,  was  intended  as  a  simple  quality  system 
requirement  for  contractors  producing  product  to  a  fully  developed  Technical  Data  Package. 
Design  of  the  product  was  complete  and  no  additional  design  was  required.  The  product 
specification  included  full  disclosure  of  the  inspection  and  test  required  to  determine 
product  acceptability.  Product  engineering  was  not  an  element  of  the  contract,  and  standard 
inspection  procedures  were  available.  Inspection  System  Requirements  is  less  stringent 
than  Quality  Program  Requirements.  Very  often  it  is  the  producer’s  first  experience  with  a 
contract  requirement  for  quality  management 

3.  Oversight  of  Contractor  Quality  Activities 

As  the  U.S.  commercial  industry  was  changing  in  the  mid-1980s,  so  too  was  the 
DoD.  Total  Quality  Management  (TQM)  had  an  office  in  OSD,  and  each  Service 
implemented  its  own  version  of  TQM.  Past  performance  started  to  become  a  selection 
factor,  and  one  saw  such  programs  as  Exemplary  Facility  and  Contractor  Performance 
Certification  Program  (CP)2.  There  was  increased  use  of  Quality  Program  Requirements 
and  an  effort  to  improve  customer/supplier  relations  between  the  government  and  the 
contractor. 

Defense  Quality  Assurance  oversight  essentially  involves  on-site  government 
personnel  performing  quality  surveillance  or  audits  of  the  contractor’s  production  activities 
related  to  contract  requirements.  The  recent  consolidation  of  all  in-plant  quality  assurance 
oversight  resources  into  the  Defense  Contract  Management  Command  (DCMC)  and  the 
introduction  of  In-Plant  Quality  Evaluation  (IQUE)  practice  provides  for  a  modem  and 
competent  capability  to  meet  the  challenge  in  the  next  century.  The  IQUE  system 
encourages  the  contractors  to  adopt  modem  quality  practices  and  innovative  means  of 
preventing  poor  quality. 

B .  THE  PROBLEM 

Current  DoD  Sampling  Inspection  Standards  include  the  following: 

•  MIL-STD-105E,  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  10  May  1989. 

•  MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection  by  Variables 
for  Percent  D^ective,  11  June  1957. 


•  MIL-STD-1235C,  Single  and  Multi-Level  Continuous  Sampling  Procedures 
for  Inspection  by  Attributes,  Functional  Curves  of  the  Continuous  Sampling 
Plans,  15  March  1988. 

MIL-STD-105  is  the  primary  document.  It  is  employed  worldwide  by  many 
nations  and  has  extensive  use  in  industry.  However,  its  need  was  established  on  what  is 
now  50-year-old  production  technology  and  techniques.  There  has  been  a  significant 
change  in  defense  products:  They  are  more  complex  and  costly,  they  contain  a  greater 
percentage  of  electixmics,  and  they  are  produced  on  automated  production  lines  with  shorter 
lead  time.  The  sampling  plans  in  MIL-STD-105  are  also  based  on  AQLs,  which  are  out  of 
step  with  best  commercial  practices  such  as  process  controls,  prevention  objectives, 
continuous  improvements,  and  parts’  reliability  of  defect  rates  measured  in  parts  per 
million. 

Sampling  plans  indexed  by  AQLs  often  had  accept/reject  criteria  that  allowed 
acceptance  of  a  lot  if,  say,  three  defects  were  found,  and  rejection  on  four  defects.  Over 
the  years,  the  use  of  these  plans  led  industry  to  believe  that  DoD  would  tolerate  less  than 
required  performance  and  gave  the  public  and  Congress  the  perception  that  DoD  would 
willingly  accept  and  pay  for  defective  product.  The  term  acceptable  quality  level  became 
onerous  and  led  to  tremendous  activity  in  the  past  eight  years  on  the  issue  of  AQLs  and  the 
intent  to  eliminate  them  from  specifications  (Table  II-l).^  The  Office  of  the  Secretary  of 
Defense  (OSD),  in  June  1987,  directed  the  removal  of  AQL  and  Lot  Tolerance  Percent 
Defective  (LTPD)  criteria  from  military  specifications  while  continuing  to  use  sampling 
procedures  with  only  accept-on-zero  (AoZ)-defects  criteria.  In  February  1989,  OSD 
directed  any  military  specifications  containing  AQLs  or  LTPDs  not  be  published.  OSD 
modem  quality  policies  allow  only  AoZ  sampling  plans.  Problems  have  developed, 
however,  because  the  engineering  and  acquisition  activities  of  defense  had  to  change  the 
way  they  had  been  doing  business  for  almost  40  years.  The  lack  of  guidance  on  using 
AoZ  plans  has  driven  contractors  and  government  QA  personnel  to  costly  100  percent 
inspection  as  a  choice  of  “what  to  do  instead.”  Woe  be  it  to  someone  who  accepted  product 
based  on  samples  containing  defects  and  the  media  found  out  about  it  Additional  guidance 
was  required. 
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A  complete  list  of  the  memoranda  issued  is  also  contained  in  the  Bibliography  of  Appendix  E. 


Table  11-1.  Previous  Efforts  to  Eliminate  Acceptable 
Quality  Levela  In  Specifications 


Date 

Action 

Issues 

16  Oct  86 

Memorandum  issued  on  "Achieving  Continuous 
Quality  Improvement’  directed  ail  OoD  specification 
oreoaration  activities  to  remove  AQLs/LTPDs 

11  Mar  87 

Memorandum  issued  reaffirming  the  16  Oct  86 
memo 

16  Jul  87 

Memorandum  directed  the  removal  of 

AQLs/LTPDs  from  Government  specifications 
while  continuing  to  use  sampling  techniques 

Raised  concern  in  the  military 
departments,  the  Defense  Logistics 
Agency  (DLA),  the  Government 

Supply  agency  (GSA),  and  industry. 
WantM  alternative  guidance  before 
aibitraiv  removal  of  AQLsA.TPDs. 

20  May  88 

Publication  of  MIL-STD  961 C.  Military 

Specifications  and  Associated  Documents, 
Preparation  of 

Specified  that  AQLs  and  LTDPs  "shall 
not  be  included  as  specification 
reauirements." 

13  Jul  88 

Final  Report  of  the  Joint  Services  Working  Group 
on  the  Elimination  of  Fixed  Defect  Levels,  which 
was  chaired  bv  Georae  Thielen 

8  Nov  88 

MIL-STD-961C,  Notice  1  issued 

Revised  statement  on  AQLs,  LTPDs: 
"Specifications  may  state  that  sampling 
inspection  for  the  purpose  of 
determining  compliance  is  acceptable. 
Fixed  AQLs  and  LTPDs  with 
associated  specific  sampling  plans, 
however,  shall  not  be  included  as 
specification  requirements.” 

12  Jan  89 

Memorandum  from  George  Thielen  with  OSD's 
response  to  workina  group's  final  report 

20  Jan  89 

Memorandum  from  HQ  AMCCOM,  AMSMO 
QAH(D),  "Recommendations  of  Working  Group  on 
Elimination  of  Fixed  Defect  Levels  from  Military 
Specifications" 

21  Feb  89 

Jack  Katzen,  OASD  TQM/SDM,  memo, 

"Elimination  of  Acceptable  Quality  Levels  (AQL's) 
and  Lot  Tolerance  Percent  Defectives  (LTPD's) 
from  Military  Specifications,"  Memorandum  for 
Assistant  Secretaries  of  the  Services  and  Director, 
DLA 

Qrders  compliance  by  30  June  1989. 

Deadline  later  postponed  and 
formation  of  DoD-wide  Process  Action 
Team  (PAT)  on  AQL/LTPD  renwval 
directed. 

3  Mar  89 

Seymour  Loiter,  AMCQA-E,  memo  setting  up 

AMC  Task  Force  chaired  bv  Geza  Pap 

16  Jun89 

Seymour  Lorber  memo:  "AMC  First-Stage  Policy 
for  the  Elimination  of  Acceptable  Quality 

Levels/Lot  Tolerance  Percent  Defectives  (AQLs, 
LTPDs)  from  Military  Specifications" 

Eliminates  AQLs/LTPDs. 

Specifies  interim  measures. 

Refers  to  Task  Force  working  on  long- 
range  plan  (to  improve  this  first-stage 
action). 

Supports  QASD  position  on 
eliminatino  AQLs/LTPDs. 
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Table  Ihl.  Previous  Efforts  to  Eliminate  Acceptable 
Quality  Levels  In  Specifications  (Continued) 


Data 

Action 

Issues 

Dec  89- 
Mav  90 

OoD-wide  Process  Action  Team  (PAT)  on 
AQL/LTPDs  removal,  chaired  bv  Stan  Beitsch 

24  Sep  90 

Final  report  of  DoD-wide  PAT  on  AQL/LTPDs 
removal 

Recommendations  ; 

Revise  MIL-STDs  961 , 962,  and  490 
to  prohibit  AQL  and  LTPD  expressions 
of  nonconformance  in  new  and 
revised  specs. 

Eliminate  AQLs  and  LTPDs  from  Mil 
Specs  during  the  normal  document 
review  cvde  (S  years). 

1 

Memorandum,  Peter  Yurcisin,  Director 
Standardization  and  Data  Management, 

OASD(P&L) 

Directing  handbook  be  prepared  tor 
specification  writers  and  Quality 
Assurance  users. 

Action;  Acting  Director,  Industrial 
Productivity  and  Quality  (John 

Todaro). 

Refers  to  DoD  PAT  convened  by 

DASD  for  TQM.  Their  final  report  — 
recommendations  -i-  methodologies  — 
basis  tor  handbook. 

15  Oct  90 

Memorandum,  John  Todaro,  OASD(P&L)PR/IPQ 

Rejects  Yurcisin  idea  as  impractical  in 
era  of  cutting  QSD  documents. 

Suggests  each  Service  carry  out 
Process  Action  Team’s  requirements. 

30  Apr  91 

General  McCausland's  letter,  DLA-Q, 
"Nonconforming  Materiar 

Reconwnends  incremental  reduction 
of  AQLs,  starting  with  all  Existing 
acceptance  numbers  greater  than  3  to 
be  reduced  by  1/3,  all  2s  reduced  to 

Is. 

17  May  91 

Memorandum.  Joceph  Pucilowski,  acting  Deputy 
Chief  of  Staff  (DCS)  for  Concurrent  Engineering 

For  Commander,  U.S.  Army  LABCQM, 
Materials  Technology  Laboratory. 
Confirmed  previous  AQL/LTPD  policy 
and  effort  to  introduce  SPC  based 
schemes  into  industry  standardization 
picture. 

^  The  efforts  to  have  better  supplier  relationships  were  not  in  line  with  the  adversarial 

I  posture  that  100  percent  inspection  implies.  Also,  increased  budget  pressures  preclude  a 

nxHe  expensive  end-item  inspection  practice  that  entails  scrap  and  rewcH-k.  The  focus  must 
[  be  on  inspection  of  the  process,  the  institution  of  process  controls,  the  prevention  of 

defective  product,  and  an  attitude  of  continuous  improvement  on  the  part  of  both 
government  and  industry. 

1 
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During  the  latter  part  of  the  1980s,  DoD  and  many  defense  contractors  discussed 
the  possibility  of  using  the  new  ISO  9000-ANSl/ASQC  90  series  of  quality  standards  in 
lieu  of  the  miliU"7  specifications.  But  DoD  representatives  did  not  see  any  significant 
benefit  in  changing  to  the  1987  version  of  the  ISO  9000  series.  Many  contractors  shared 
the  same  doubts  as  the  military.  As  a  result,  no  changes  were  made  at  that  time. 


III.  DEVELOPING  NEW  QUALITY  ASSURANCE 
PRACTICES  FOR  DOD 


Spurred  by  increasing  pressure  to  change  its  way  of  doing  business,  DoD 
approached  the  Institute  for  Defense  Analyses  (IDA)  to  conduct  a  study  that  would 
ultimately  lead  to  new  quality  assurance  practices  for  Defense.  IDA  was  approached 
because  of  its  vast  previous  experience  with  concurrent  engineering  and  quality 
management  practices  within  industry.  The  subject  was  controversial  because  of  high 
Congressional  visibility  and  a  public  perception,  based  on  sampling  inspection  plans  still  in 
place,  that  DoD  was  willing  to  accept  lots  that  contained  defects.  Objectivity  and 
independence  were  required  to  develop  a  new  standard  practice  that  both  DoD  and  industry 
would  be  ccnnfortable  with. 

A.  IDA  APPROACH  AND  FINDINGS 

IDA  set  out  to  analyze  the  best  commercial  practices  in  order  to  recommend  changes 
to  DoD  quality  assurance  practices.  IDA  also  enlisted  the  help  of  recognized  acceptance 
sampling  and  quality  assurance  experts.  Dr.  Ed  Schilling,  a  professor  at  the  Rochester 
Institute  of  Technology  and  author  of  numerous  books  on  acceptance  sampling,  and  Mr. 
Seymour  Lorber,  retired  Deputy  Chief  of  Concurrent  Engineering  for  Army  Materiel 
dlommand,  were  hired  as  consultants. 

1.  Sampling  Inspection  Standards 

IDA  collected  information  on  quality  standards  and  systems  used  in  both  defense 
and  commercial  industry.  Appendix  A  contains  a  list  of  the  more  prominent  dociunents  and 
activities  in  this  area,  and  Appendix  E  contains  the  entire  Inbliography  of  literature  reviewed 
during  this  study.  IDA  spoke  to  quality  assurance  professionals  in  both  commercial  and 
defense  industry  and  sent  questionnaires  to  industrial  organizations  dealing  with  product 
standards,  asking  about  their  use  of  AQLs  for  product  acceptance  sampling  in  their 
standards  and  their  views  on  the  use  of  nongovernment  standards  (NGSs)  by  the  DoD. 
Their  responses  are  recorded  in  the  documented  briefing  in  Appendix  B.  This  information 
related  to  the  AQL  elimination  initiatives  within  DoD  and  its  move  toward  greater  use  of 
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industry  standards.  IDA  also  documented  the  series  of  activities  in  DoD  to  eliminate  the 
use  of  AQLs  in  military  standards,  as  was  shown  in  Table  IM  of  the  last  chapter.  The 
current  mandate  is  that  inspection  sampling  be  conducted  using  only  accept-on-zero-defects 
plans.  IDA  found  that  the  best  commercial  practices  in  industry  have  demonstrated  that 
high  quality  and  lower  cost  can  be  achieved  not  by  end-item  inspection  but  by  control  of  the 
processes  that  produce  the  end-item.  Instead  of  “inspecting  quality  in”  after  production, 
resulting  in  rework  and  scrap  and  the  additional  costs  they  entail,  process  controls  aim  to 
minimize  scrap  and  rework  by  the  prevention  of  defects. 

One  could  reasonably  ask,  “Does  DoD  require  sampling  inspection  inspection 
procedures  at  all?” — a  question  to  which  the  answer  is  clearly  “yes”  when  one  considers 
DoD  as  a  customer.  It  buys  large  quantities  of  many  items  from  many  producers  and  many 
suppliers.  DoD  policies  require  competition,  interchangeability,  standardization,  and  repair 
parts  for  the  life  of  the  systems.  In  this  environment,  a  uniform  means  of  establishing 
characteristics  is  required,  but  minimum  verification  costs  are  desired.  Some  formula  for 
government-prime- subcontractor  communications  needs  to  be  devised. 

Some  initial  IDA  findings  and  observations  about  sampling  inspection  procedures 
included  the  following: 

•  For  selected  commodities,  there  may  be  no  need  to  change  the  sampling 
inspection  practices;  however,  the  military  standards  could  be  replaced  with 
their  commercial  equivalent. 

•  The  description  of  the  characteristics  of  the  product  (Critical,  Major,  Minor) 
should  be  retained  in  the  product  specification. 

•  A  standard  sampling  procedure  could  be  prepared  using  selected  zero-defects 
acceptance  plans. 

•  A  new  procedure  based  on  Statistical  Process  Control  (SPC)  concepts  for 
selected  application  could  then  be  phased  in. 

These  findings  led  IDA  to  prepare  a  draft  of  a  new  quality  assurance  standard  for 
DoD  and  work  with  a  recognized  industry  association  SPC  committee  with  industry, 
academia,  and  DoD  participants  to  develop  the  standard  and  get  it  accepted. 

2.  Quality  Management  System  Specifications 

As  discussed  in  Chapter  I,  quality  management  and  process  controls  have  been 
recognized  worldwide  for  the  competitive  advantage  they  offer.  Commercial  application  of 
the  ISO  9000  series  has  expanded  in  the  U.S.  and  the  world.  Industry  has  voluntarily 
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changed  its  approach  to  quality,  no  longer  viewing  a  quality  system  as  an  onerous  military 
requirement.  Contractors  have  recognized  the  benefit  of  prevention  for  controlling  costs  as 
well  as  meeting  major  (including  military)  buyers’  requirements  for  quality  systems.  If 
DoD  were  to  continue  to  insist  that  all  defense  contractors  comply  with  the  military 
specification,  these  contractors  would  then  have  to  maintain  documentation  for  Quality 
Program  Requirements  as  well  as  for  ISO  9000  in  order  to  do  business  commercially  or 
internationally.  This  would  be  a  costly  process,  limiting  the  cost  advantages  DoD  might 
obtain  by  encouraging  dual  use  facilities. 

Comparing  Quality  Program  Requirements  with  the  requirements  of  ISO  9001 
revealed  minimal  differences  except  for  paragraph  4.4,  Design  Control,  which  covers 
quality  system  requirements  during  design  phases.  Quality  Program  Requirements  does 
not  mention  design  control.  However,  all  defense  contractors  doing  development  and 
engineering  respond  to  a  wide  variety  of  technical  requirements  for  contract  management  of 
product  development. 

For  these  reasons,  IDA  recommended  that  DoD  not  only  authorize  use  of  the  ISO 
9000  series  but  aggressively  plan  for  its  full  adoption  over  a  reasonable  period  of  time. 
With  this  authorization,  defense  should  not  endorse  or  require  certification  by  a  third 
independent  party.  The  restriction  does  not  prevent  the  contractor  from  using  consultant  or 
other  services  to  implement  a  quality  program.  With  regard  to  this  recommendation,  it  is 
noted  that  OSD,  on  14  February  1994,  authorized  the  use  of  the  ISO  9000-ANSI/AS(5C 
Q90  series  for  new  programs  and  as  appropriate  for  ongoing  programs.'  The  use  of  a 
single  quality  system  in  a  facility  “provides  for  cost  effective,  high  quality  products  and 
services  and  improved  process  capability .”2 

IDA  found  that  the  ISO  series  cumently  does  not  provide  a  document  comparable  to 
Inspection  System  Requirements.  ISO  9002,  Quality  Systems — Model  for  Quality 
Assurance  in  Production  and  Installation,  is  essentially  similar  to  9001  except  that 
paragraph  4.4,  Design  Control,  is  not  included.  The  similarity  between  9001  and  9002 


'  John  Deutch,  Under  Secretary  of  Defense,  Memorandum  for  Secretaries  of  the  Military  Departments, 
Directors  of  Defense  Agencies,  Use  of  Commercial  Standards  in  the  Department  of  Defense  (DoD), 
14  February  1994. 

^  Walter  B.  Bergmann  11,  Acting  Assistant  Secretary  (Production  Resources),  MIL-HDBK-9000, 
Guidance  in  the  Application  of  ISO  9000~ANSI/ASQC  Quality  System  Requirements,  Memorandum 
for  Assistant  Secretary  of  the  Army  for  Research,  Development  and  Acquisition,  Assistant  Secretary  of 
the  Navy  for  Research,  Development  and  Acquisition,  Assistant  Secretary  of  the  Air  Force 
(Acquisition),  and  Directors  of  Defense  Research,  14  February  1994. 
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suggests  that  in  the  future,  9002  may  be  abandoned  and  contractors  will  be  catalogued  as 

9001  with  or  without  design  control  as  appropriate.  ISO  9003,  Quality  Systems — Model 
for  Quality  Assurance  in  Final  Inspection  and  Test,  is  essentially  equivalent  to  the  standard 
inspection  clause  required  to  be  included  in  all  U.S.  government  contracts. 

Inspection  System  Requirements  is  not  able  to  meet  the  current  demand  for 
improved  manufacturing  pra>  ::es  and  does  not  take  advantage  of  the  current  quality 
management  environment  It  is  an  inspection  document  that  does  not  provide  an  emphasis 
on  prevention  and  control.  The  number  of  contractors  currently  using  Inspection  System 
Requirements  is  substantial,  and  the  number  can  be  expected  to  grow  as  industry  generally 
upgrades  their  quality  system.  For  example,  a  DCMC  review  found  that  in  the  plants 
where  they  perform  in-plant  quality  assurance  activities,  800  facilities  meet  Quality 
Program  Requirements,  7200  facilities  meet  Inspection  System  Requirements,  and  nearly 
8000  facilities  work  to  the  simple  standard  inspection  clause  requirements.  Using  ISO 

9002  or  ISO  9003  in  place  of  Inspection  System  Requirements  does  not  produce  the 
desired  result  and,  unfortunately,  places  a  signiHcant  burden  on  the  majority  of  industry 
supplying  DoD  (much  of  the  this  industry  is  small  business).  IDA  determined  that  the 
standard  should  not  only  provide  a  replacement  for  the  current  sampling  standards,  with  a 
new  approach  to  accomplish  strict  sampling  inspection,  and  require  manufacturing  process 
and  statistical  process  controls  for  the  most  critical  product  characteristics,  but  also  provide 
an  incentive  to  producers  to  use  effective  quality  practices  and  process  controls. 

3.  Oversight  of  Contractor  Quality  Activities 

One  of  the  IDA  findings  on  best  commercial  practices  is  that  relationships  with 
suppliers  should  be  based  on  trust  with  audit  rather  than  on  extensive  product  inspection. 
The  consolidation  of  all  in-plant  quality  assurance  oversight  resources  into  the  DCMC  and 
the  introduction  of  IQUE  practice  have  substantially  improved  relations  with  defense 
contracu^rs.  Early  in  the  task,  IDA  met  with  Mr.  Ernest  Ellis,  Deputy  Executive  Director, 
Quality  Assurance,  and  his  staff  at  the  Defense  Logistics  Agency  (DLA)  and  presented  a 
approach  for  the  IQUE  program  to  move  finom  inspection  sampling  to  acceptance  of  product 
based  on  a  producer’s  SPC  data.  This  approach,  called  the  “ABC’  process,  was  developed 
by  Carmen  Liuzza  and  Paul  Roediger  of  the  (Quality  Assurance  Directorate  of  the  Armament 
Research,  Development,  and  Engineering  Center  at  Picatinny  Arsenal,  NJ,  and  Ed  Shilling 
of  the  Rochester  Institute  of  Technology.  The  Isiefing  given  to  DLA  was  well  received  and 
is  contained  in  Appendix  C,  A  Three-Stage  Sampling  Plan  to  Attain  Process  Control  and 
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Capability.  It  was  originally  thought  that  this  ABC  process  could  be  developed  into  the 
recommended  standard.  Although  useful  to  the  IQUE  people  at  DLA,  the  standard  took  a 
different  approach.  In  its  present  form,  however,  the  draft  standard  is  compatible  with  the 
IQUE  approach. 

B  .  NEW  DRAFT  STANDARD 

IDA  developed  the  original  concept  of  an  SPC  quality  assurance  plan  on  the  basis 
of  contractor  requirements  in  Quality  Program  Requirements  and  the  use  of  SPC  in 
accordance  with  the  American  Society  for  Quality  Control  and  American  National 
Standards  Institute  ANSI  Zl. 1-1985,  Guide  for  Quality  Control  Charts,  ANSI  Zl.2-1985, 
Control  Chart  Method  of  Analyzing  Data,  and  ANSI  Zl.3-1985,  Control  Chart  Method  of 
Controlling  Quality  During  Production.  Processes  were  to  be  established  to  meet  full 
production  capabilities,  and  when  a  process  fell  below  requirements,  an  aggressive  effort  to 
improve  was  to  be  required.  Production  may  have  been  restricted  until  required  levels  were 
achieved.  The  procurement  contracting  officer  (PCO)  and  the  acquisition  contracting 
officer  (ACO)  were  to  cooperate  with  the  contractor  to  achieve  required  levels.  The 
proposed  process  capability  indices  (Cpks)  and  types  of  sampling  required  at  the  various 
stages  for  critical,  major,  and  minor  characteristics  of  the  product  that  were  to  be  required 
are  as  shown  in  Table  III-l. 


Table  III-l.  Cpk  Values  for  Critical,  Major,  and  Minor 
Characteristics  at  Different  Phases 


Initial  Plan 

Initial  Production 

Full  Production 

Critical 

Major 

Minor 

1.33  (100%  Screening) 

1.33  (A-o-Z  Sampling) 

1.00  (A-o-Z  Sampling) 

1.66  (A-o-Z  Sampling) 

1.33  (A-o-Z  Sampling) 

1.00  (A-o-Z  Sampling) 

2.00  (0  Sampling) 

1.66  (0  Sampling) 

1.33  (0  Sampling) 

The  draft  standard  underwent  many  changes.  IDA  began  working  with  the 
American  Defense  Preparedness  Association  (ADPA)  SPC  Division  Technical  Committee 
in  an  effort  to  get  industry,  academia,  and  government  support  for  the  standard.  The 
quality  assurance  professionals  on  this  committee  are  listed  in  Table  in-2.  Concepts  for  the 
standard  changed  along  the  way  as  consensus  was  reached  within  the  committee.  IDA  also 
sent  the  various  versions  out  to  additional  government  and  industry  quality  assurance  and 
standards  personnel  for  further  informal  coordination.  Appendix  D  contains  the  new  draft 
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standard  developed  under  this  task.  This  document  was  developed  as  a  possible  way  to 
move  DoD  procurement  quality  assessment  requirements  from  lot-by-lot  sampling  toward 
process  controls  and  continuous  improvement.  Following  current  mandate,  it  contains 
accept-on- zero-defects  sampling  tables,  but  that  is  not  its  primary  objective. 


Table  111-2.  ADPA  SPC  Technical  Committee  Members 


Name 

Organization 

Mr.  Geza  Pap,  Chair 

U.S.  Army  ARDEC,  Picatinny  Arsenal,  NJ 

Mr.  James  Childress 

Army  Management  Engineering  College,  Rock  Island,  IL 

Mr.  Robert  M.  Chvatal 

COMARCO,  Bloomfield,  IN 

Mr.  Ray  Ediund 

U.S.  Anny  HQ,  AMCCOM,  Rock  Island,  IL 

Dr.  Donald  S.  Ermer 

University  of  Wisconsin,  Madison,  WS 

Mr.  Bob  Formella 

ARMTEC  Defense  Systems,  Coachella,  CA 

Mr.  Raymond  Hamblin 

Alliant  TechSystems,  Brooklyn  Park,  MN 

Dr.  Anand  Joglekar 

Alliant  TechSystems,  Brooklyn  Park,  MN 

Mr.  Dan  Kedzie 

COMARCO,  Bloomfield,  IN 

Ms.  Jennifer  Kibiger 

Olin  Ordnance,  St.  Petersburg,  FL 

Mr.  Carmen  Liuzza 

U.S.  Army  ARDEC,  Picatinny  Arsenal,  NJ 

Mr.  Seymour  Lorber 

Consultant,  Institute  for  Defense  Analyses,  Alexandria,  VA 

Mr.  Bill  Mitrik 

Olin  Ordnance,  St.  Petersburg,  FL 

Mr.  Harlan  Patterson 

Hughes  Aircraft  Co.,  El  Segundo,  CA 

Dr.  Karen  J.  Richter 

Institute  for  Defense  Analyses,  Alexandria,  VA 

Mr.  Greg  Stein 

BMY-Combat  Systems,  HARSCO  Corp,  York,  PA 

Dr.  Ken  Tieman 

Loral  Aeronuttonic,  Newport  Beach,  CA 

Mr.  Rich  Zerilli 

Defense  Logistics  Agency,  Alexandria,  VA 

The  document  is  intended  to  replace  the  AQL-based  existing  military  standards,  and 
intentionally  avoids  the  usual  statistical  details  and  methodologies.  It  provides  a  sampling 
procedure  based  on  zero  acceptance  criteria  for  attributes  sampling  and  comparable  plans 
for  variables  and  continuous  sampling.  The  hundreds  of  pages  in  the  three  primary 
inspection  sampling  standards  are  reduced  to  three  simple  accept-on-zero-defects  tables 
with  straightforward  examples.  Detailed  rules  for  switching  between  plans  are  provided, 
based  on  the  results  of  the  inspection.  The  objective  is  to  create  an  atmosphere  where  every 
nonconformance  is  an  opportunity  for  corrective  action  and  improvement  rather  than  one 
where  AQLs  are  the  contractually  sufficient  goals. 
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The  two  key  features  of  this  document  are  its  Alternate  Acceptance  Provisions, 
which  establish  incentives  for  initiating  preventative  quality  programs,  and  fairly  high 
sampling  producer’s  risks.  They  are  intended  to  complement  each  other,  such  that 
suppliers  of  high  quality  goods  can  gain  product  acceptance  on  the  methods  they  are  using 
to  achieve  the  quality  (SPC,  etc.),  while  suppliers  of  marginal  quality  are  confionted  by 
increased  screening  and  administrative  costs.  The  underlying  theme  is  a  partnership 
between  the  supplier  and  DoD,  with  the  requisite  competence  of  both  parties,  and  a  clear 
mutual  benefit  for  processes  capable  of  consistently  high  quality  product 

The  draft  standard  was  put  into  the  required  format^  and  is  now  undergoing  formal 
coordination  among  the  Services  as  a  MIL-STD,  sponsored  by  the  Army  at  Picatinny 
Arsenal.  The  draft  standard  is  also  being  considered  as  a  commercial  standard. 


^  Yes,  there  is  a  standard  on  how  to  write  standards.  Preparation  of  Military  Standards,  Handbooks,  and 
Bulletins,  MIL-STD-%2B,  20  May  1988. 
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IV.  CONCLUSIONS 


This  task  demonstrated  an  approach  toward  acquisition  reform  that  involved 
government,  industry,  and  academia  working  together  with  IDA’s  help  to  do  something 
that  OSD  can  tackle  on  its  own  without  requiring  Congressional  action.  IDA  is  continuing 
to  work  for  the  acceptance  of  the  new  standard  by  analyzing  final  review  comments  and  is 
assisting  in  the  development  of  a  handbook  to  accompany  the  standard  with  the  ADPA  SPC 
Division.  This  handbook  must  address  both  how  to  develop  a  Technical  Data  Package 
(TDP)  with  the  standard  and  what  the  government  and  contractor  must  do  when  the 
contract  cites  the  standard.  And,  of  course,  IDA  continues  to  advise  OSD  on  execution  of 
the  new  quality  assuranceAnanagement  practices. 

A.  RESULTS  AND  CONSEQUENCES 

Adoption  of  the  draft  Standard  under  the  recommendations  given  below  and  the 
authorization  of  the  ISO  9(X)0-ANSI/ASQC  Q90  series  together  help  facilitate  a  common 
approach  and  DoD  use  of  best  commercial  specifications  and  practices.  The  use  of  a  single 
quality  system  helps  allow  the  dual  use  of  facilities  in  the  move  toward  the 
commeicial/military  integration  and  the  development  of  a  national  industrial  base.  The 
lower  cost  products  capable  under  this  new  quality  management  approach  will  help  to  have 
globally  competitive  defense  contractors  and  a  govemment/contractor  relationship  that 
encourages  improved  process  control  and  a  culture  of  continuous  improvement 

B.  RECOMMENDATIONS 

1 .  Improving  Defense  Acquisition  Quality  Assurance  Activities 

Efforts  by  DoD  during  the  past  several  years  to  improve  its  acquisition  quality 
assurance  operation  have  resulted  in  many  signiHcant  improvements.  In  the  interest  of 
further  improvement,  IDA  recommends  that  DoD  continue  to  increase  procurement  of 
commercial  products,  reduce  government  oversight  of  contractors,  utilize  commercial 
specifications  and  standards  wherever  possible,  strive  for  the  prevention  of  defects  by  the 
implementation  of  integrated  product  and  process  development  (IPPD),  and  acconunodate 
dual-use,  commercial/militaiy  integration  in  production  lines. 
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Specific  to  the  task  described  in  this  paper,  IDA  recommends  the  following; 

•  Replace  MIL-STD-105,  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  with  its  commercial  equivalent,  ANSI/ASQC  Zl.4-1993. 

•  Continue  to  use  ANSI/ASQC  Zl.4-1993  plans  for  products  that  are  of 
relatively  simple  design,  described  by  a  complete  specification,  and  for  which 
experience  has  demonstrated  no  problems. 

•  Replace  MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection  by 
Variables  for  Percent  D^ective,  with  its  commercial  equivalent,  ANSI/ASQC 
Zl.9-1993. 

•  Request  ASQC  to  develop  an  ANSI/ASQC  standard  to  replace  MIL-STD- 1235 
and  cancel  the  MIL-STD. 

•  Replace  MIL-I-4S208,  Inspection  System  Requirements  with  the  new  draft 
standard,  DoD  Preferred  Methods  for  Acceptance  of  Product  (Appendix  D  to 
this  paper). 

2.  Related  Activities  in  DoD 

While  this  effon  was  initiated  in  1991,  it  has  dovetailed  with  recommendations  in 
two  recent  activities.  The  joint  Military  and  National  Aeronautical  and  Space 
Administration  (NASA)  Handbook,  Interim  Guidance  on  the  Application  of  ISO 
9000/ ASQC  Q90  Series  Quality  Systems  in  Standards,  was  issued  on  4  February  1994. 
Its  purpose  is  “to  assist  contracting  activities  that  have  decided  to  use  Q91  or  (^2  quality 
system  standards  with  domestic  contracts  and  seek  guidance  for  doing  so.” 

In  April  1994,  the  Report  of  the  Process  Action  Team  on  Military  Specifications 
and  Standards  was  also  issued.  It  makes  recommendations  for  a  process  focus,  AQL 
elimination,  and  reduced  contractor  inspection  and  test.  Specifically,  under  Oversight,  the 
report  recommends  two  specific  tasks  related  to  the  draft  standard  and  the  recommendations 
in  this  paper 

•  “DepSecDef  issues  a  policy  memorandum  emphasizing  greater  use  of  process 
controls  in  lieu  of  development  and  production  testing  and  inspection.”^ 


^  Report  of  the  Process  Action  Team  on  Military  Specifications  and  Standards,  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology,  April  1994,  p.  100. 
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•  “Develop  a  priority  action  list  of  military  specifications  containing  fixed 
allowable  defect  level  measures  such  as  acceptable  quality  levels  or  lot 
tolerance  percent  defect.  Initiate  action  to  eliminate  requirements  for  these 
defect  measures.”^ 

And  under  Contractor  Test  and  Inspection,  the  report  states:  ‘The  contractor  shall  certify  to 
the  government  that  the  item  or  items  offered  for  acceptance  and  delivery  satisfy  the 
requirements  of  the  specifications  through  process  controls  and  inspections.”^ 

The  draft  standard  in  Appendix  D  fits  with  these  two  recent  activities  and  should  be 
approved  with  the  rectxmnended  elimination  of  the  other  documents  listed  in  Section  B.  1. 

3.  Summary  of  Recommendations 

a.  Sampling  Inspection  Requirements 

•  Cancel  MIL-STD-105,  MIL-STD-414  and  MIL-STD-1235,  Single-  and  Multi- 
Level  Continuous  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  Functional  Curves  of  the  Continuous  Sampling  Plans. 

•  Use  ANSI  standards  equivalent  with  canceled  standards. 

•  Urge  ASQC  to  prepare  an  ANSI  standard  to  replace  MIL-STD- 1 235. 

b.  Quality  Management  Standards 

•  Implement  aggressive  efforts  to  replace  MIL-Q-9858  with  ISO  9001- 
ANSI/ASQCQ91. 

•  Replace  MIL-I-45208  with  a  new  standard,  DoD  Pr^erred  Methods  for 
Acceptance  of  Product  (Appendix  D  to  this  report). 

4.  "Defect  Anticipation”  Follow-On  Effort 

This  study  had  as  its  focus  the  encouragement  of  improvements  in  defect  prevention 
and  process  control  as  a  means  for  improved  quality  in  DoD.  To  make  truly  significant 
quality  (both  performance  and  cost  effectiveness)  gains,  the  challenge  is  for  I>oD  to 
develop  defect  anticipation  practices.  These  are  efforts  that  influence  the  product  designs  to 
inpx>ve  their  robustness  and  tolerance  to  the  vagaries  of  the  production  floor  as  well  as  the 


2  Ibid. 

3  Ibid.  p.  107. 
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battlefield  environment.  When  defect  anticipation  and  variability  reduction  come  together, 
the  prospects  of  Process  Based  Acceptance  may  become  real. 
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Appendix  A 

QUALITY  AND  INSPECTION  SAMPLING  STANDARDS 


A.  DEFENSE  STANDARDS 
MILITARY  STANDARDS  (MIL-STDS) 

MIL-STD-1 05  SampSng  Procedures  and  T^les  for  Inspection  by  Attributes 

Inspection  by  attributes  is  inspection  whereby  either  the  unit  of 
product  is  classified  sirnpiy  as  defective  or  non^jefective  or  the 
number  of  defects  in  the  unit  of  product  is  counted  with  respect  to 
a  given  requirement  or  set  of  re^iremeias. 

Attributes  sampling  plans  have  the  advantage  of  greater  simplicity, 
of  being  applick)le  to  either  single  or  multipie  quality 
characteristics,  and  of  requiring  no  knowledge  about  the 
distribution  of  the  continuous  measurements  of  any  of  the  quality 
characteristics. 

MIL-STO-109  QuaHty  Assurance  Terms  and  Deriruthns 

MIL-STO-414  Samfriing  Procedures  and  Tables  for  Inspection  by  Variables  for 

Percent  Defective 

The  variables  sampling  plans  apply  to  a  single  quality  characteristic 
that  can  be  measured  on  a  continuous  scale  and  for  which  quaiity  is 
expressed  in  tenns  of  percent  defective.  The  theory  underlying 
the  development  of  the  variables  sampling  plans,  including  the 
operating  characteristic  (OC)  curves,  assumes  the  measurements 
of  the  quality  characteristics  are  independent,  identically 
distributed,  normal  random  variables. 

In  comparison  with  attributes  sampling  plans,  variables  sanpling 
plans  have  the  advantage  of  usually  resulting  in  oonsiderabie 
savings  in  sample  size  for  comparable  assurance  as  to  the 
correctness  of  decisions  in  judging  a  single  quality  characteristic  or, 
tor  the  same  sample  size,  greater  assurance  is  obtained  using 
variables  plans. 

MIL-STD-1 2^  Single,  and  Multi-level  Continuous  SampSng.  Procedures  and 

Tables  for  Insperriion  by  Attributes. 

MILITARY  SPECIFICATIONS  (MIL-SPECS) 

MIL-l-45208  Inspection  System  Requimments 

MIL-Q-9858  Quatty  Program  Requimments 

MILITARY  HANDBOOKS  (MIL-HDBKS) 

MIL-HDBK-53-1  Guide  for  Attribute  Lot  Sarttping  Inspection  and  MIL-STD- 105. 

Portions  copied  from  ISO  2859-1974,  Addendum  1  (1977). 
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MIL-HDBK-53-2  Guide  tor  Attrtoute  Con^nuous  Sampling  Inspection  and  MIL-STD- 

1235 

MIL-HDBK-53-3  Guide  tor  Variables  Lot  Sampling  Inspection  and  MiL-STD-414 

TECHNICAL  REPORTS 

TR-7  Factors  and  Procedures  for  Applying  MIL-STD-1  OSD  Sampling 

Plans  to  Life  and  Reliability  Testing. 

DoD  Quaiity  Control  and  Reliability  Assurance  Tech  Report,  OASD 
(Supply  and  Logistics),  1965. 


B .  COMMERCIAL  STANDARDS 

AMERICAN  NATIONAL  STANDARDS  INSTITUTE/AMERICAN  SOCIETY  FOR 
QUALITY  CONTROL  (ANSI/ASQC) 

ANSI  Z1 .1/ASQC  B1  Guide  tor  Quality  Control  Charts 

(1985) 

ANSI  Z1 .2/ASQC  B2  Control  Chart  Methods  of  Analyzing  Data 

(1985) 

ANSI  Z1 .3/ASQC  B3  Control  Chart  Method  of  Controlling  Quality  During  Production 
(1985) 

ANSI  Z1 .15  (1979)  Generic  Guidelines  tor  QuaMty  Systems 

ASQC  Z1 .4  (1981 )  Sarr^ang  Procedures  and  Tables  tor  Inspection  by  Attributes. 

Equivalent  to  MIL-STD-IOS. 

ASCQC  Z1 .9  (1 982)  Sampling  Procertores  and  Tables  for  Inspection  by  Variables,  for 

P{ercent  Nonconforming. 

Equivalent  to  MIL-STD-414. 

ANSI/ASQC  A1  (1987)  Definitions,  Symbols,  Formulas  and  Tables  for  Control  Charts 

ANSI/ASQC  A2  (1987)  Terms,  Syrrtools,  and  Defiriirions  tor  Acceptance  Sampling 

ANSI/ASQC  A3  (987)  Quality  Systems  Temvnology 

ASQC  Cl  (1968)  General  Requirements  fora  Quality  Program 

ASQC  E3  (1984)  Guide  to  Inspection  Planting 

ASQC  090  (1987)  Quatity  Management  and  Quality  Assurance  Standards^ 

Guidelines  for  Selection  and  Use 

Provides  guidelines  for  the  selection  and  use  of  Standards  Q91 , 
092,  Q93,  and  Q94. 

ASQC  Q91  (1987)  Quatity  Systems— Model  tor  Quatity  Assurance  in 

Design/Development,  Production,  Installation  and  Senridng 

Specifies  quality  system  requirements  for  use  where  a  contract 
between  two  parties  requires  the  demonstration  of  a  supplier's 
capability  to  design  and  supply  product. 
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ASCX^  Q92  (1987) 


QuaSty  Systems — Model  for  Quality  Assurance  in  ProrhJCtion  and 
Inst^lation 

Specifies  quaiity  system  requirements  for  use  where  a  contract 
between  two  parties  requires  the  demonstration  of  a  suppiier's 
capability  to  control  the  processes  that  determine  the  acceptability 
of  a  product  supplied. 

ASCX^  Q93  (1987)  QuaBty  Systems— Model  for  Quality  Assurance  in  Final  Inspection 

and  Test 

Specifies  quality  ^stem  requirements  for  use  where  a  contract 
between  twc  parties  requires  the  demonstration  of  a  supplier's 
capability  to  design  and  control  the  disposKion  of  any  product 
nonconformity  during  final  inspection  and  test. 

ASQC  Q94  (1987)  QuaUty  Management  and  Quality  System  Elements— Guidelines. 

Describes  a  basic  set  of  elements  by  which  a  Quality  Management 
System  can  be  developed  and  implemented  internally. 

AMERICAN  NATIONAL  STANDARDS  INSTITUTE/INSTITUTE  FOR  INTER¬ 
CONNECTING  AND  PACKAGING  OF  ELECTRONIC  CIRCUITS  (ANSI/IPC) 

ANSI/IPC-PC-90  (1990)  General  Requirements  for  Implementation  of  Statistical  Process 

Control 

AMERICAN  SOCIETY  OF  MECHANICAL  ENGINEERS  (ASME) 

ASM  E  F  AP-1  ( 1 990)  Quality  Assurance  P/pg/am  Requirements  for  Fastener 

Manufacturers  and  DistritMJtors 

AMERICAN  SOCIETY  FOR  TESTING  MATERIALS  (ASTM) 

ASTM  STD  1 5D  Manual  on  PresentaVon  of  Data  and  Control  Chart  Analysis 

ASTM  El  05  (1 958)  Recommended  Practice  for  Probability  Sampling  of  Materials 

ASTM  El  22  (1972)  Recommended  Practice  for  Choice  of  Sample  Size  to  Estimate  the 

Average  Quality  of  a  Lot  or  Process 

ASTM  El 41  (1 969)  Recommended  Practice  for  Acceptance  of  Evidence  Based  on  the 

Results  of  ProbatxSty  Sampling 

ASTM  E456  (1 983)  Terminology  for  Statisecal  Methods 

AMERICAN  NATIONAL  STANDARDS  INSTITUTE/ELECTRONICS  INDUSTRIES 
ASSOCIATION  (ANSI/EIA) 

ANSI/EIA-557  (1989)  Statistical  Process  Control  Systems 

ANSI/EIA-584  (1991 )  Zero  Acceptance  Number  SampHr^  Procedures  and  Tables  for 

Inspection  by  Attributes  of  a  Continuous  Manufacturing  Process 

ANSI/EIA-585  (1991)  Zero  Acceptance  Number  Sampling  Procedures  and  Tables  for 

Attributes  of  Isolated  Lots 

ANSI/EIA  IS-17  (1985)  Assessment  of  Outgoing  Defective  Levels  in  PPM 
JEDEC  No,  19  (1988)  General  Stemdsud  for  Statistical  Process  Control  (SPC) 

BOEING 

DI-9000  (1991)  Advanced  Quality  System 
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FORD 

Q-101  (1990)  Worldwide  Quality  System  Standard  for  Manufacturing  Operations 

and  Outside  Suppliers  of  Production  and  Service  Products 

Quality  System  Survey  arxl  Scoring  Guidelines,  15  Apr  90. 

Worldwide  Supplier  Quality  Rating  System,  15  Apr  90. 

The  Initial  Sample  Review  Process  lor  Suppliers  to  Ford  Motor 
Company,  15  Apr  90. 

Planning  for  Quality.  15  Apr  90. 

Q1  01  Preferred  Quality  Award  for  Suppliers  to  Ford  Motor 
Company,  15  Apr  90. 

Facilities  and  Tools,  Quality  System  Standard,  May  1991 
Q-1 01 W  (1 991 )  Worldwide  Quality  System  Standard.  Warehouse  and  Distritsution 

FORD.  CHRYSLER,  AND  GENERAL  MOTORS 

Fundamental  Statistical  Process  control.  Reference  M-anual-l99l 

Measurement  Systems  Analysis.  Reference  Manual-1990 
QuaSty  System  Standard  (Draft)- 1993 

INDUSTRIAL  FASTENERS  INSTITUTE 

Recommended  Practices  for  Statistical  Process  Control 


INTERNATIONAL 

ISO  3534  (1977) 
ISO  8402  (1966) 

ISO  9000(1987) 

ISO  9001  (1987) 

ISO  9002  (1987) 

ISO  9003  (1987) 

ISO  9004  (1987) 
ISO  2859  (1974) 

ISO  3951 
lEC 

lEC  410  (1973) 
(Recommendation) 

lECQ 


STANDARDS  ORGANIZATION  (ISO) 

Statistics— Vocabulary  and  Symbols 

Quafity— Vocabulary 

Referenced  in  all  of  ANSi/ASQC  90  series 

QuaSty  Management  and  QuaSty  Assurance  Standards— 
Guidelines  tor  Selection  and  Use 

Equivalent  to  ASQC  Q90  series. 

QuaSty  Systems— Model  for  QuaSty  Assurance  in 
Design/Development.  Production.  Installation  and  Senndng 

QuaSty  Systems— Model  for  QuaSty  Assurance  in  Production  and 
Installation 

QuaSty  Systems— Model  for  Quality  Assurance  in  Final  Inspection 
and  Test 

QuaSty  Management  and  QuaSty  System  Elements— Guidelines 

SampSng  Procedures  and  Tades  for  Inspection  by  Attributes.  Parts 
1  and  2 

International  version  of  MIL-STD-IOSD 
Intemationat  graphical  vemion  of  MIL-STD-414 

SampSng  Plans  and  Procedures  for  Inspection  by  Attributes 
lEC  QuaSty  Assessment  System  for  Electronic  Components 
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QC  001001  (1981) 


Basic  Rules  of  the  lEC  Quality  Assessment  System  for  Electronic 
Component 


TECHNICAL  COMMITTEES 
ISO/TC176  Quality  Assurance 

Wrote  ISO  9000  series 


isorrc69 


ANSI/ASQC  Z-1 


AppUcaHons  of  Statistical  Methods 

Wrote  ISO  Standards  Handbook  3,  Statistical  Methods,  as  well  as 
other  standard  guides  and  codes  of  practice 

QuaSty  Assurance 
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The  list  of  industry  standards  associations  from  which  to  solicit  information  was  derived  from  the  listing  of  indusi 
groups  in  the  Standardization  Directory  (SD-1)  of  the  Defense  Standardization  and  SpeciHcation  Program.  Industry  groups  w( 
chosen  if  their  name  implied  that  they  might  be  more  product  than  process  oriented  and  more  likely  to  have  standards 
specifications  for  product  inspection,  llie  list  is  as  follows: 
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INTERESTING  RESULTS 

When  the  comments  are  seen  attributed  to  the  specific  standards  organization,  it  appears  as  though  those  with  the  greatest 
defense  applicatitMi  are  those  most  in  favor  of  keeping  AQLs.  Almost  all  organizations  favored  the  greater  use  of  NGSs  by  DoD. 
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INTERESTING  RESULTS  (Cont’d) 
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Appendix  C 

A  THREE-STAGE  SAMPLING  PLAN 
TO  ATTAIN  PROCESS  CONTROL 
AND  CAPABILITY 
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Initial  Screening 


Contractor  enters  stage  C  if  product  passes  the  screening. 


The  first  stage  represents  the  inspect  quality  into  the  product  approach.  This  represents  screening  and  stage  C. 
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stage  C 


Contractor  enters  stage  B  if  Cpk>1.0  and  last  lot  had  no 
defects 


Stage  B  takes  the  form  of  a  mixed  variable-attribute  plan,  the  mechanics  of  v^hich  are  depicted  in  the  figure.  The  terms 
accept"  and  "reject"  refer  to  disposition  of  the  product  insp^ted,  whereas  "pass"  and  "fail"  refer  to  status  of  the  sampling  plan 
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Advance  to  level  BO  when  Cpk>1.2,  R  chart  in  control. 


M  M  O 

<<gs 

J'  4)  o  S 

u  EU.S 

1 S 

o  fcs 

g';^c.s-n 

i)o's-^^ 

<«  o5  ^  O  fi 

*S  S**®  «  ?i 

alS’i  s 

^3  PS  ^  N 

1  S  5 

^  -I  ®  =  iJ 

§l-sll 

C  -c  .12  ^ 

5?  <*«  f<i  - . 

s.S  c.2^ 
e  & 

«  c0  12 
f>  c  ’O  cx  i» 

a*«  S-a  « 
js  c  .s  S  .5 

o  V  a  X  ^ 
p'i  «  e| 

0  *;  g  „  "a 
o  .§,.SP-S  S. 
^  c  e  o 

<s  i  $  u  di 

iiiii 

E  u  cd  .M  ^ 

Sills 

“*•03  “  S* 

>0  §’§  c  E 

®  X  "c  s  a 

nin 

ca  w  s  c 

if  §11 

^  "S  E  S 
2  «  i  'C 

=ll  s 

£  S-S  E 


C-20 


Level  BO 


a  as 

OJ 

S  c  iQ 

5  •“  © 

a-  gC 

S  -2*8 
a  w  t3 

^  5  « 

S  42 
S  S  g 

'S  a<.S 

O 


c« 

©  g  M 

•S  ©  © 

wf^ 

•nr*^  ’7? 
<2 

e  .V  a 

S  a  S 

So® 

•r  W)  C 


h  ^ 
S  j2 
<jJ  "5- 


^  CIS 

*s?5 

-  ^ 
s  ^ 

c2  fi 
®  fi 

*s  ^ 
«5 

^  U 

^  a 


.  DC 

^  .S 


CI4  ^ 

S  X. 

es  es 

OJ  ^ 


.S  «  is 

C 

CQ  O  I 

u  ^  X 


a 

o 

W3  « 

S 

B  *1^ 

•S  ^  ^ 

©  *^  .s 

fe.  S  s 

J  5  s 


Advance  to  stage  A  when  Cpk>1.33. 


< 

9i 

01) 


•g-g«2 

*o  52 

IH 

« s 

1*1 

Si's  *5 

•c  o  ^ 
s  •^’3 

«*  -pH  C 

.Sac 

8 

O  4>  “ 

I1« 

<u 

a  j>  t3 

S'B  g 

«>  c  . 

«>  **•3 

111 


M  . 

P 


fe—  o 
g*o.c 
a  g-- 

M  O  C 

£  o-g 
-.sS 

^<i 

S  p  s 

8  &  o 

C  M  0^ 

w.S'g 
.S  -  § 

>.^•9 

fix 

W9  O  o 

.  .  C  CQ 

1f1 

o.  P 

*-  <  o 
o.  s 
P  2 
8 

8s  u 

•55  O 

s-gg 

•s  S'k 

u  Otis 
ep  w>  A 

§-il 

ire 

e  y  u 

V.S£ 

^s^.g 

-fJB  C 

-1*2 
S  C 

8 


u 

9 


I 


C-22 


stage  A 


01) 

CQ 


Must  be  in  control  with  Cpk>1.33  to  remain  in  A. 


e-S.S 

ii  ^  J 

m 

V  u 

■S  M 

i§i 

•s  fg 

®  aco 

4>  CA 

&  O  M 

IIS 

*”  V  C 

=aS8 


|g§ 

•I  hiJ 

*0  S  *> 

§1^ 

—  S'® 

g”“  a  e« 

®'C 
C  (A-s 

8§a 

s « s 

A 

o<^ 
‘Oft  S 

«  s-s 

M  c  e 

8‘^i 

C  Q  V 
S  *9  o 

•2  ’S.e 

so  s 

rtv-J 


nut^ 

“vc  loie'SS 

ifiaei-S  Smase 


i  §1^  5-1  b 

.s  -S  ^ 

•S'S’tt’SirS  e' 


mn 

.S  a  5  o  — 


8i‘6*l  =^s.l 

uum 

■ljil  gl  *1 

iseS»“^S 

X o<5  S  h3.M 

S*>'8'-2  S 
8  a-s  E 1  s  I 

^  C  ..  2  _ 


•3  5  8  -a  •a 

a-n  Ca  u 

Sg-sls 

l<lll 

?“til 


ll^-Elp 

■a|§'i5  it 

5  V  3'  *2  .  «  § 

<-i§  §  s-sl 
.  Q  u  .s  ^  s 
g>^  w>2  «  J  2 
.5  .  *>  S  .2  « *9 
js  w  e^ts  S  S 
3  Pm  ^  ^  ^ 


•S  s  3  -o  I.  h 

Ji§§gf-i 

B  s2  C  O  A 

M  —  4J  —  g  w  Q 

2  p,-3  e  0  -'5 
•  ^  a  Q  MM 
Be  “ *3  a 
i2  i  3  S  S  «  - 


ii?'P 

5<pa^-l 

1  i>i  SC’S 

O  T> 

^  O  Q  c  C  O 

§Hf.  8..2  2 

^siS  SE 

g.^5‘21  g 

a“-s>2  2i 

<  o  ez:  s  .5 

®  g  K 

p-i  giioQ  e 

SUMS 

^  8  I  ^  e  «> 


3  c  2L,  4. 

“r? 

B  e 

is  3’2  ® 

0-202 

<§t^ 

1- si 

2- -2  S. 

61 S 


—  2  ^  S*-S  ’3  8 
’S'S’S  fi  S 
2.3  B  S*f  g-S 

’^§’3  i  oil 

•3  «  g  g  o  -3  i 

llllll« 

aSsi2i« 

iliiP 
I  lull 


C  ®  I  JB  1*9' 

■gts's^l 

E  .2  olg  J  PQ 

B  sl'S  s 

(A  .3  a  3  8>S 

S  (B  P  O  S  p 

bP-B  .«  «» 

a  S2‘'s  i 

<^*='20  C  a 

®  o  S-M  O’® 

'S  |B  T3  '8  a 

1lll& 

S  ^8  8 


C-24 


Rules 


O 


cs  s 

■■3.2 

2  •- 
S-o 

S  § 


e 

o 

Z  § 

S  £ 


2  — 
£  OA  S 

c-S  'o 

a>  ^  O 
2:2  ^ 
m  I 


ti  ^ 

•S  Cta 
*©  © 
a<  *o 

<3 


pS  ^ 

£  > 

L« 

O 

o  ^ 

tt  2 

5  S 

^  c 
.S 

in  ^ 

O  -fi 

c  2 
o  z 

^  S  e 

^  i2 

^  e  © 

^  .s  © 

o  o  a 
®  Q, 

s 

©  o  •■* 

DC  O 
^  12  © 
fi  S  gc 

•g  fi  iS 
&(  ^  ^ 

CD  ^  S 

-S  ®  © 


E  oc 

e  ^ 

CD  2 

*5 

o*  & 

u  2 

'I  a 

pS  CD 

s  o 
© 

C->  S 
^  2 
s 


C-25 


Anned  with  a  quality  plan  and  preceded  by  the  screening  phase,  ABC  looks  like  the  figure. 
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ABC  Summary 
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FOREWORD 


1 .  This  standard  is  approved  for  use  by  the  Department  of  Defense,  Defense  Contractors, 
and  other  commercial  organizations. 

2.  This  standard  provides  a  set  of  sampling  plans  and  procedures  for  planning  and 
conducting  the  inspection  of  product  to  assess  quality  and  conformance  to  contract 
requirements.  This  standard  complies  with  the  Department  of  Defense  G^D)  policy  of 
eliminating  acceptable  quality  levels  (AQLs)  and  associated  practices. 

3.  The  following  points  provide  the  basis  for  this  standard: 

•  Defense  contractors  are  required  to  submit  product  that  conforms  to 
requirements  and  to  generate  and  maintain  sufficient  evidence  of  conformance. 

•  Contractors  are  responsible  for  establishing  their  own  manufacturing  and 
process  controls  to  produce  product  in  accordance  with  requirements. 

•  Contractors  are  expected  to  use  conunon  industrial  practices  such  as  process 
controls  and  statistical  techniques. 

•  Department  of  Defense  (DoD)  procurement  practices  encourage  industry 
innovation  and  provide  flexibility  to  achieve  the  benefits  of  improvement 

4.  Sampling  inspection  is  a  common  industrial  practice  for  demonstrating  the 
conformance  of  product  to  the  requirements  of  the  contract  and  its  technical  data 
package.  The  application  of  sampling  plans  for  acceptance  involves  both  consumer 
and  producer  risks.  Increased  sampling  is  one  way  of  reducing  these  risks,  but  it  also 
increases  costs.  Producers  can  reduce  risks  by  employing  effective  processes  with 
appropriate  process  controls.  To  the  extent  that  such  practices  are  employed  and  are 
effective,  risk  is  controlled  and,  consequently,  product  inspection,  including  testing, 
can  be  reduced. 
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5.  Manufacturing  process  controls  and  statistical  control  methods  are  the  preferable 
means  of  preventing  nonconformances,  controlling  quality,  and  generating  informadon 
for  product  improvement.  An  effective  process  control  system  may  also  be  used  to 
provide  informadon  to  assess  the  quality  of  product  submitted  for  acceptance.  The 
suppliers  are  encouraged  to  use  process  control  and  stadsdcal  control  procedures  for 
their  internal  control  and  to  consider  submitdng  effective  process  control  procedures  in 
place  of  prescribed  sampling  requirements  to  the  government  fca*  approval. 
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1.  SCOPE 


1.1  PURPOSE 

This  standard  establishes  the  DoD  preferred  set  of  sampling  plans  and  procedures 
for  the  acceptance  of  product 

1.2  APPLICABILITY 

This  standard,  when  referenced  in  the  contract,  specification,  or  purchase  order,  is 
applicable  to  all  suppliers  at  the  contractor,  subcontractor,  or  vendor  facilities.  The 
sampling  plans  shall  be  applied  as  specified  in  the  contract  documents,  and  product  may  be 
submitted  for  acceptance  if  die  requirements  of  this  standard  have  been  met 

1.3  PRODUCT  REQUIREMENTS 

The  contractor  is  required  to  submit  pxxluct  that  meets  all  contract  and  specificatitm 
requirements.  The  application  of  sampling  plans  in  this  standard  does  not  relieve  the 
contractor  of  responsil^ity  for  meeting  all  cmitract  product  requirements.  The  contractor’s 
quality  system,  including  manufacturing  processes  and  quality  control  measures,  shall  be 
established  and  operated  to  consistently  produce  products  that  meet  all  requirements. 
Absence  of  any  inspection  or  process  control  requirement  in  the  contract  shall  not  relieve 
the  contractor  of  responsibility  for  assuring  that  all  products  or  supplies  submitted  to  the 
government  for  acceptance  conform  to  all  requirements  of  the  contract 

1.4  APPLICATIONS  AND  LIMITATIONS 
1.4.1  Applications 

Sampling  plans  and  procedures  in  this  standard  when  appropriate  may  be  used  to 
assess  conformance  to  requirements  of  the  following: 

•  End  items 

•  Conqxxients  or  basic  materials 

•  Operations  or  services 
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•  Materials  in  process 

•  Supplies  in  storage 

•  Maintenance  {^rations 

•  Data  or  records 

•  Administrative  procedures 

1.4.2  Limitations 

The  sampling  plans  and  procedures  of  this  standard  are  not  intended  for  use  with 
destructive  tests  or  where  product  screening  is  not  feasible  or  desirable.  In  such  cases,  the 
sampling  plans  will  be  stated  elsewhere  in  the  contract  or  product  speciHcations. 


D-2 


2.  APPLICABLE  DOCUMENTS 


2.1  MILITARY  STANDARDS 

MIL-STD-109,  Quality  Assurance  Terms  and  Derations 

2.2  COMMERCIAL  STANDARDS 
ISO  8402,  Quality-Vocabulary 
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3.  DEFINITIONS  AND  TERMS 


The  deflnitions  and  terms  in  ISO  8402  and  the  following  are  applicable.  When 
terms  and  definitions  listed  below  or  in  the  contract  and  supporting  reference  requirements 
differ  from  those  in  ISO  8402,  they  will  take  precedence  over  ISO  8402. 

3.1  DEFINITIONS  FROM  FEDERAL  ACQUISITIONS  REGULATION 
(FAR)  46.101 

3.1.1  Acceptance 

The  act  of  an  authorized  representative  of  the  Government  by  which  the 
Government,  for  itself  or  as  agent  of  another,  assumes  ownership  of  existing  identified 
supplies  tendered  or  approves  specific  services  rendered  as  partial  or  complete  performance 
of  the  contract 

3.1.2  Contract  Quality  Requirements 

The  various  functions,  including  inspection,  performed  by  the  Government  to 
determine  whether  a  contractor  has  fulfilled  the  contract  obligations  pertaining  to  quality 
and  quantity. 

3.1.3  Government  Contract  Quality  Assurance 

The  various  functions,  including  inspection,  performed  by  the  Government  to 
determine  whether  a  contractor  has  fulfilled  the  contract  obligations  pertaining  to  quality 
and  quantity. 

3.1.4  Inspection 

Examining  and  testing  supplies  or  services  (including,  when  appropriate,  raw 
materials,  components,  and  intermediate  assemblies)  to  determine  whether  they  conform  to 
contract  requirements. 


D-4 


3.1.5  Off-The-Shelf  Item 

An  item  produced  and  placed  in  stock  by  a  contractor,  or  stocked  by  a  distributor, 
before  receiving  orders  or  contracts  for  its  sale.  The  item  may  be  commercial  or  produced 
to  military  or  Federal  specifications  or  description. 

3.1.6  Subcontractor 
(sec  44.101) 

3.1.7  Testing 

That  element  of  inspection  that  determines  the  properties  or  elements,  including 
functional  operation  of  supplies  or  their  components,  by  the  application  of  established 
scientific  principles  and  procedures. 

3.2  DEFINITION  FROM  DEPARTMENT  OF  DEFENSE  FEDERAL 
ACQUISITION  REGULATIONS 

3.2.1  Metrology 

The  science  of  weights  and  measures  used  to  determine  conformance  to  technical 
requirements  including  the  development  of  standards  and  systems  for  absolute  and  relative 
measurements. 

3.2.2  Quality 

The  composite  of  material  attributes  including  performance  features  and 
characteristics  of  a  product  or  service  to  satisfy  a  given  need. 

3.2.3  Quality  Assurance 

A  planned  and  systematic  pattern  of  all  actions  necessary  to  provide  adequate 
confldence  that  adequate  technical  requirements  are  established;  products  and  services 
conform  to  established  technical  requirements;  and  satisfactory  performance  is  achieved. 

3.2.4  Quality  Audit 

A  systematic  examination  of  the  acts  and  decisions  with  respect  to  quality  in  order 
to  independently  verify  or  evaluate  the  operational  requirements  of  the  quality  program  or 
the  specification  or  contract  requirements  of  the  product  or  service. 
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3.2.5  Quality  Program 

A  program  which  is  developed,  planned,  and  managed  to  carry  out  cost  effectively 
all  efforts  to  effect  the  quality  of  materials  and  services  from  concept  through  validation, 
full-scale  development,  production,  deployment,  and  disposal. 

3.3  CLASSIFICATION  OF  CHARACTERISTICS 

The  enumeration  of  characteristics  of  product,  classified  according  to  their 
importance.  Characteristics  will  normally  be  grouped  into  classes  of  critical,  major,  or 
minor,  however,  they  may  be  grouped  into  other  classes  or  subclasses  within  these  classes. 

3.4  CRITICAL  CHARACTERISTIC 

A  characteristic  that  judgment  and  experience  indicate  must  be  met  to  avoid 
hazardous  or  unsafe  conditions  for  individuals  using,  maintaining,  or  depending  upon  the 
product;  or  that  judgment  and  experience  indicate  must  be  met  to  assure  performance  of  the 
tactical  function  of  a  major  item  such  as  a  ship,  aircraft,  tank,  missile,  or  space  vehicle. 

3.5  MAJOR  CHARACTERISTIC 

A  characteristic,  other  than  critical,  that  must  be  met  to  avoid  failure  or  material 
reduction  of  usability  of  the  unit  of  product  for  intended  purpose. 

3.6  MINOR  CHARACTERISTIC 

A  characteristic,  other  than  critical  or  major,  whose  departure  from  its  specification 
requirement  is  not  likely  to  reduce  materially  the  usability  of  the  unit  of  product  for  its 
intended  purpose  or  whose  departure  from  established  standards  has  little  bearing  on  the 
effective  use  or  operation  of  the  unit. 

3.7  NONCONFORMANCE 

A  departure  from  a  specified  requirement  for  any  characteristic. 

3.8  NONCONFORMING  UNIT 

A  unit  of  product  that  has  one  or  more  nonconformances. 
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3.9  CRITICAL  NONCOr  FORMING  UNIT 

A  unit  of  product  that  fails  to  conform  to  specified  requirements  for  one  or  more 
critical  characteristics. 

3.10  MAJOR  NONCONFORMING  UNIT 

A  unit  of  product  that  fails  to  conform  to  specified  requirements  for  one  or  more 
major  characteristics,  but  conforms  to  all  critical  characteristics. 

3.11  MINOR  NONCONFORMING  UNIT 

A  unit  of  product  that  fails  to  conform  to  specified  requirements  of  one  or  more 
minor  characteristics,  but  conforms  to  all  critical  and  major  characteristics. 

3.12  SCREENING  INSPECTION 

An  inspection  process  where  eveiy  unit  is  checked  and  all  nonconfonning  units  arc 
removed;  also  referred  to  as  100  percent  inspection. 

3.13  PRODUCTION  INTERVAL 

Normally  is  a  single  shift;  it  can  be  a  day  if  it  is  reasonably  certain  that  shift  changes 
do  not  affect  quality  of  product,  but  shall  not  be  longer  than  a  day. 

3.14  VERIFICATION  LEVEL  (VL) 

Prescribes  the  level  of  significance  or  utility  of  a  characteristic  to  the  user.  The 
amount  of  effort  to  assure  conformance  can  be  allocated  on  the  basis  of  importance  to  the 
user.  (Major  characteristics  will  require  more  verification  effort  than  minor  characteristics.) 
VL-VII  requires  the  highest  level  of  effort,  and  the  effort  decreases  as  the  VL  decreases  to 
the  lowest  level,  VL-I. 
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4.  SAMPLING  INSPECTION  REQUIREMENTS 


4.1  PREFERRED  SAMPLING  PLANS 

This  standard  establishes  three  sets  of  matched  sampling  plans  for  the  sampling 
inspection  of  product  submitted  to  the  government  for  acceptance.  These  sampling  plans 
provide  for  inspecting  the  samples  from  lots  or  batches  by  attributes  or  variables 
measurement  and  for  continuous  sampling  by  attributes  measurement.  The  three  sets  of 
matched  sampling  plans  are  indexed  by  seven  specified  verification  levels  (VL)  and  five 
code  letters  (CL),  which  are  determined  by  the  lot  or  production  interval  size.  The 
sampling  plans  are  matched  between  corresponding  VL  and  CL  combinations  to  result  in 
essentially  similar  producer’s  risk.  The  contractor  may  utilize  the  type  of  plan,  at  the  same 
verification  level,  that  best  complements  the  production  process. 

4.2  FORMATION  AND  IDENTIFICATION  OF  LOTS  OR  BATCHES 

The  product  shall  be  assembled  into  identifiable  lots,  sublots,  or  batches,  or  in  such 
other  manner  as  may  be  prescribed.  Each  lot  or  batch  shall,  as  far  as  practicable,  consist  of 
unit  of  product  of  a  single  type,  grade,  class,  size,  and  composition,  manufactured  under 
essentially  the  same  conditions,  and  at  essentially  the  same  time.  The  lots  or  batches  shall 
be  identified  by  the  contractor  and  shall  be  kept  intact  in  adequate  and  suitable  storage 
space.  Although  lot  or  batch  size  is  not  used  to  select  a  continuous  sampling  plan,  the 
formation  of  lots  or  batches  may  remain  desirable  for  reasons  of  homogeneity,  shipping 
convenience,  and  facilitation  of  payment 

4.3  DETERMINATION  OF  SAMPLING  PLAN 

A  sampling  plan  is  determined  by: 

•  Verification  level  (VL)  as  specified. 

•  Type  of  sampling  (attributes,  variables,  or  continuous). 

•  Lot  or  production  interval  size  code  letter  (CL)  from  Table  1 ,  Section  4.8.1. 

•  Switching  procedure  (normal,  tightened,  reduced). 
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For  lot  accq>tance  situations  (attributes  or  variables),  the  occurrence  of  one  or  more 
nonconformances  shall  result  in  withholding  acceptance  of  the  product  submitted  and 
initiation  of  corrective  action.  When  continuous  sampling  is  in  effect,  the  occurrence  of  a 
nonconforming  unit  while  in  a  sampling  phase  results  in  withholding  acceptance  of  that 
unit,  a  return  to  screening,  and  initiation  of  corrective  action.  If  a  nonconforming  unit  is 
found  while  in  a  screening  phase,  acceptance  is  withheld  for  that  unit  and  screening  is 
continued  until  the  requirements  of  paragraph  4.9.3.2  are  satisified. 

4.4  SAMPLING  OF  LOTS  OR  BATCHES 

4.4.1  Selection  of  Units 

Units  of  product  drawn  from  a  lot  for  a  sample  shall  be  selected  at  random  from  the 
lot  without  regard  to  their  quality.  Random  sampling  requires  that  each  unit  in  the  lot, 
batch,  or  production  interval  have  the  same  probability  of  being  selected  for  the  sample. 

4.4.2  Representative  (Stratified)  Sampling 

When  appropriate,  the  number  of  units  in  the  sample  shall  be  selected  in  proportion 
to  the  size  of  sublots  or  subbatches,  or  parts  of  the  lot  or  batch,  identified  by  some  rational 
criterion.  When  representative  sampling  is  used,  the  units  from  each  sublot,  subbatch,  or 
pan  shall  be  selected  at  random. 

4.4.3  Process  of  Sampling 

A  sample  may  be  drawn  after  all  units  comprising  the  lot  or  batch  have  been 
assembled,  or  sample  units  may  be  drawn  during  assembly  of  the  lot  or  batch,  in  which 
case  the  size  of  the  lot  or  batch  will  be  determined  before  samples  are  drawn.  When  the  lot 
or  batch  passes  the  sampling  plan,  such  lots  or  batches  are  acceptable  and  may  be  submitted 
to  the  government.  When  sample  units  are  drawn  during  lot  or  batch  assembly  and 
nonconforming  units  are  found,  the  contractor  shall  withhold  from  acceptance  that  portion 
of  the  lot  completed  and  all  additional  production  occurring  prior  to  the  initiation  and 
verifrcation  of  corrective  action.  For  lots  or  batches  withheld  from  acceptance,  the 
ccHitractor  shall  take  the  following  actions: 

•  Screen  the  lots  or  batches  and  dispose  of  all  nonconforming  units  in 
accordance  with  paragraph  4.5. 

•  Determine  the  cause  of  the  nonconformances  and  implement  appropriate 
process  changes. 
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•  Initiate  the  switching  requirements  of  paragraph  4.8.3. 

•  Advise  the  government  representative  of  actions  taken  and  sulxnit  the  screened 
lot  or  batches  to  the  government 

4.5  DISPOSITION  OF  NONCONFORMING  PRODUCT 

All  units  of  product  found  to  be  nonconforming  by  the  contracts  shall  be  removed 
and  kept  apart  fiom  the  flow  of  production  or  otherwise  identified  ot  segregated  to  preclude 
submission  to  the  government.  The  contractor  may  rework  or  repair  these  units  unless  the 
contract  excludes  such  activities.  Corrected  product  will  be  screened  by  the  contractor  and 
resubmitted  to  the  government  apart  finom  the  regular  flow  of  the  product 

4.6  SPECIAL  RESERVATIONS  FOR  CRITICAL  NONCONFORMANCE 

When  a  critical  nonconformance  is  discovered  at  any  phase  of  production  or  during 
any  inspection,  the  following  immediate  action  is  required:  prevent  delivery  of  critical 
nonconforming  units  to  the  government,  notify  the  government  representative,  screen  all 
available  units,  and  take  corrective  action.  Records  of  corrective  actions  shall  be 
maintained  and  made  available  to  the  government  representative. 

4.7  CRITICAL  CHARACTERISTICS 

For  each  critical  characteristic,  the  contractor  is  required  to  implement  an  automated 
screening  or  a  fail  safe  manufacturing  operation  and  apply  sampling  plan  VL- VII  to  verify 
the  performance  of  the  screening  operation  unless  otherwise  specified  in  the  contract  or 
product  specifications.  The  occurrence  of  one  or  more  critical  nonconformances  requires 
corrective  action  as  specified  in  paragraph  4.6. 

4.8  SAMPLING  INSPECTION 

4.8.1  Verification  Level  Specification 

The  VLs  are  specified  in  the  contract  or  product  specifications.  A  VL  may  be 
specified  for  individual  characteristics,  for  a  group  of  characteristics,  or  for  subgroups  of 
characteristics  within  the  group.  The  VL  and  code  letter  (CL)  from  Table  i  determine  the 
sampling  plan  required  to  assess  product  compliance  to  contract  ana  speciHcation 
requirements.  Contractors  are  expected  to  produce  and  submit  product  in  full  conformance 
to  all  requirements.  Lots,  batches,  or  productitm  intervals  of  product  that  consistently  meet 
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or  exceed  all  requirements  will  be  accepted  by  the  sampling  plans  of  this  standard  and  will 
result  in  qualifying  for  reduced  sampling  levels. 


Table  1.  Code  Letters  (CL)  for  Entry  Into  the  Sampling  Tables 


Lot  or  Production 

Verification  Levels 

Interval  Size 

VII 

VI 

V 

IV 

1 

1 

2-170 

n 

B 

B 

Bi 

B 

A 

A 

171-288 

WM 

WBM 

WBM 

WBM 

A 

B 

289-544 

■n 

mM 

mM 

mm 

B 

C 

545-960 

B 

BB 

A 

A 

B 

C 

D 

961-1632 

■1 

WBM 

A 

B 

C 

D 

E 

1633-3072 

■■ 

mm 

B 

C 

D 

E 

E 

3073-5440 

A 

B 

C 

D 

E 

E 

E 

5441-9216 

B 

C 

0 

E 

E 

E 

E 

9217-17408 

C 

D 

E 

E 

E 

E 

E 

17409-30720 

D 

E 

E 

E 

E 

E 

E 

30721  and  larger 

E 

E 

E 

E 

E 

E 

E 

4.8.2  Sampling  Procedures 

Unless  otherwise  described,  the  VL  specified  in  the  contract  shall  be  considered  the 
normal  level  of  inspection  and  will  be  used  at  the  start  of  inspection.  Normal,  tightened,  or 
reduced  sampling  inspection  shall  continue  unchanged  for  each  group  of  characteristics  or 
individual  characteristic  except  where  the  switching  procedures  given  in  paragraph  4.8.3 
require  change.  The  switching  procedures  shall  be  applied  to  each  group  of  characteristics 
or  to  individual  characteristics. 

4.8.3  Switching  Procedures 

The  sampling  plan  criteria  for  normal,  tightened,  and  reduced  inspection  are  given 
in  Tables  2, 3,  and  4  (Note  2  to  the  respective  table)  of  Section  4.9. 

The  switching  procedures  are  independent  of  the  results  oi  any  remedial  action, 
such  as  screening,  additional  samples,  etc.,  resulting  from  the  occurrence  of  sample 
ncMiconformances  and  withholding  of  acceptance. 

Some  Table  4  switching  criteria  depend  upon  a  corresponding  Table  2  entry.  These 
entries  have  been  denoted  by  na(N)  and  na(T)  in  the  descriptions  that  follow.  na(N) 
represents  the  Table  2  sample  size  used  for  normal  sampling  at  the  VL  and  (X  currently  in 
effect.  Likewise,  na(T)  represents  the  tightened  sample  size. 
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4.8.3. 1  Normal  to  Tightened 

When  normal  inspection  is  in  effect,  tightened  inspection  shall  be  instituted  when 
one  of  the  following  conditions  occurs,  depending  on  the  type  of  sampling  plan  being 
used. 

•  Lot  or  batch  sampling  (Tables  2  and  3): 

—  2  lots/batches  have  been  withheld  from  acceptance  within  the  last  5  or 
fewer  lots/batches. 

•  Continuous  sampling  (Table  4): 

—  2  nonconforming  units  are  found  within  the  last  5  segments  of  size  na(N), 
or  fewer,  units  inspected. 

4. 8. 3. 2  Tightened  to  Normal 

When  tightened  inspection  is  in  effect,  normal  inspection  may  be  instituted  when 
the  following  condidtvis  are  both  satisfied. 

•  The  cause  for  producing  the  nonccHtformances  is  corrected. 

•  Lot  or  batch  sampling  (Tables  2  and  3): 

—  5  consecutive  lots/batches  are  accepted. 

Continuous  sampling  (Table  4): 

—  The  last  5  segments  of  size  na(T)  units  inspected  contain  only  consecutive 
conforming  units. 

4. 8. 3. 3  Normal  to  Reduced 

When  normal  inspection  is  in  effect,  reduced  inspection  may  be  instituted  when  the 
following  conditions  are  all  satisfied. 

•  Lot  or  batch  sampling  (Tables  2  and  3): 

—  10  consecutive  lots/batches  are  accepted  while  on  normal  inspection. 
Continuous  sampling  (Table  4): 

—  The  last  10  segments  of  size  na(N)  units  inspected  contain  only 
consecutive  conforming  units. 

•  Production  is  at  a  steady  rate. 

•  The  contractor’s  quality  system  is  considered  satisfactory  by  the  government 

•  Reduced  inspection  is  considered  desirable  by  the  government. 
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4. 8. 3. 4  Reduced  to  Normal 


When  reduced  inspection  is  in  effect,  normal  inspection  shall  be  instituted  when  the 
following  conditions  occur. 

•  Lot  OT  batch  sampling  (Tables  2  and  3): 

—  A  lot/batch  is  withheld  from  acceptance. 

Continuous  san^ling  (Table  4): 

—  A  nonconforming  unit  is  found. 

•  Production  becomes  irregular  or  delayed. 

•  The  contraaor’s  quality  system  is  unsatisfactory. 

•  Other  conditions  warrant  that  normal  inspection  be  instituted. 

4. 8. 3. 5  Discontinuation  of  Acceptance 

If  sampling  inspection  of  lots  or  batches  remains  in  tightened  inspection  due  to 
discovery  of  nonconformances  or  when,  on  continuous  sampling  plans,  there  are  long 
periods  of  screening  due  to  discoveiy  of  nonconformances,  the  government  reserves  the 
right  to  discontinue  acceptance  of  the  product  until  the  causes  of  nonconformances  are 
eliminated  or  other  means  acceptable  to  the  procuring  agency  have  been  instituted.  When 
sampling  inspection  is  restarted  after  discontinuation  of  acceptance,  it  shall  be  at  the 
tightened  inspection  level. 

4.9  PREFERRED  SAMPLING  INSPECTION  TABLES 

See  Appendix  A  for  methods  of  computing  sampling  results,  using  switching  rules, 
and  detemuning  compliance  with  requirements  using  the  attributes,  variables,  and 
continuous  sampling  plans  contained  in  this  section. 

4.9.1  Attributes  Sampling  Plans  for  Lot  or  Batch  Inspection 

The  preferred  attributes  sampling  plans  for  lots  or  batches  are  described  in  Table  2 
for  normal,  tightened,  and  reduced  inspection. 
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Tabto  2.  Attributes  Sampling  Plans 


Verificatbn  Levels 

Code 

Letter 

T 

Vll 

VI 

V 

IV 

1 

1 

1 

R 

Sample  Size  (na) 

A 

3072 

1280 

512 

192 

80 

12 

5 

■■ 

B 

4096 

1536 

640 

256 

96 

16 

6 

MM 

C 

5120 

2048 

768 

320 

128 

48 

20 

8 

D 

6144 

2560 

1024 

384 

160 

64 

24 

10 

E 

8192 

3072 

1280 

512 

192 

80 

32 

12 

NOTES: 

(1)  When  the  bt  size  is  bss  than  or  equal  to  the  sampb  size,  100  percent  attributes  inspectbn  is 
required. 

(2)  One  verifbation  level  (VL)  to  the  left/right  of  the  specified  normal  VL  is  the  respective 
tightened/reduced  plan.  Tightened  inspectbn  of  VL-VII  is  T,  reduced  inspectbn  of  VL-I  is  R. 

(3)  The  bt  acceptability  criteria  is  that  the  sampb  shall  contain  nonconformances. 


4.9.2  Variables  Sampling  Plans  for  Lot  or  Batch  Inspection 

The  preferred  variables  sampling  plans  for  lots  or  batches  are  described  in  Table  3 
for  normal,  tightened,  and  reduced  inspection. 

4.9.2. 1  Limitations  on  Use  of  Table  3 

Table  3  is  not  to  be  used  indiscriminately.  Its  use  shall  depend  upon  evidence, 
provided  by  graphical  or  statistical  analyses,  that  the  assumptions  of  independence  and 
normality  are  being  met.  Table  2  shall  be  used  whenever  the  evidence  fails  to  warrant  use 
of  Table  3. 

4. 9. 2. 2  Nonconforming  Unit 

For  the  purposes  of  variables  sampling,  a  unit  of  product  for  which  the  variables 
measurement  exceeds  the  specified  tolerance  is  considered  as  a  nonconforming  unit.  One 
or  mcne  nonconforming  units  in  the  sample  shall  be  cause  for  withholding  acceptance  of  the 
lot  or  batch. 
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Table  3.  Variables  Sampling  Plans 


Verification  Levels 

Code 

Letter 

T 

VII 

VI 

■■ 

IV 

1 

1 

1 

R 

Sample  Size  (ny) 

A 

mm 

87 

64 

44 

18 

9 

4 

B 

B 

wm 

92 

69 

49 

20 

11 

5 

C 

la 

100 

74 

54 

37 

23 

13 

7 

WM 

D 

136 

107 

81 

58 

41 

26 

15 

8 

BI 

E 

145 

113 

87 

64 

44 

29 

18 

9 

B 

k  Values  (One- 

or  Two-Sided) 

A 

3.51 

3.27 

3.00 

2.69 

2.40 

2.05 

1.64 

lEI 

1^ 

B 

3.58 

3.32 

3.07 

2.79 

2.46 

2.14 

1.77 

Wm 

C 

3.64 

3.40 

3.12 

2.86 

2.56 

2.21 

1.86 

Bi 

1.20 

D 

3.69 

3.46 

3.21 

2.91 

2.63 

2.32 

1.93 

1.56 

1.20 

E 

3.76 

3.51 

3.27 

3.00 

2.69 

2.40 

2.05 

1.64 

1.21 

F  Values  (Two-Sided) 

A 

mgm 

msm 

■01 

.222 

.271 

.370 

.707 

B 

m 

wm 

■g 

■g 

HI 

.214 

.253 

.333 

.707 

C 

.132 

M| 

wm 

WM 

.182 

.208 

.242 

.301 

.707 

D 

.130 

wm 

m 

.162 

.177 

.199 

.233 

.283 

.435 

E 

.128 

111 

.145 

.157 

.174 

.193 

.222 

.271 

.370 

NOTES: 


(1)  When  the  lot  size  is  less  than  or  equal  to  the  sample  size,  100  percent  attributes  inspection  is 
required. 

(2)  One  verification  level  (VL)  to  the  left/right  of  the  specified  normal  VL  is  the  respective 
tightened/reduoed  plan.  Tightened  inspection  of  VL-VII  is  T,  reduced  inspection  of  VL*I  is  R. 

(3)  The  lot  acceptability  criterion  is  that  the  sample  shall  contain  no  nonconformances  and  shall  also 
meet  the  appli(»ble  k  and  F  acceptability  criteria  described  in  Appendix  A  (Examples  2  and  3). 
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4.9.3  Continuous  Attributes  Sampling  Inspection  Plans 

The  preferred  continuous  sampling  plans  for  inspection  by  attributes  are  described 
in  Table  4  for  normal,  tightened,  and  reduced  inspection. 

Table  4.  Continuous  Sampling  Plans 


Code 

Letter 

T 

Verification  Levels 

R 

VII 

VI 

n 

IV 

1 

1 

1 

Screening  Phase:  Clearance  Numbers  (I) 

A 

3867 

2207 

1134 

527 

264 

125 

55 

27 

NA 

B 

7061 

3402 

1754 

842 

372 

180 

83 

36 

NA 

C 

11337 

5609 

2524 

1237 

572 

246 

116 

53 

NA 

D 

16827 

6411 

3957 

1714 

815 

368 

155 

73 

NA 

E 

26912 

11868 

5709 

2605 

1101 

513 

228 

96 

NA 

Sampling  Phase:  Frequencies  (f) 

A 

1/3 

2/17 

Ba 

■B 

1/34 

1/48 

B 

4/17 

ng 

1/12 

■SB 

wm 

1/48 

1/68 

C 

1/6 

1/17 

1/24 

Hal 

1/68 

1/96 

0 

2/17 

1/12 

1/17 

1/24 

1/34 

Bl 

KIBH 

1/96 

1/136 

E 

1/12 

1/17 

1/24 

1/34 

1/48 

HI 

1/96 

1/136 

1/192 

NOTES: 


(1)  Use  of  other  i  and  f  combinations  are  permined  provided  they  are  computed  in  accordance  with 
Appendix  A,  Example  5. 

(2)  One  verification  level  (VL)  to  the  left/right  of  the  specified  normal  VL  is  the  respective 
tightened/feduced  plan.  Tightenad  inspection  of  VL-VII  is  T,  reduced  inspection  of  VL-I  is  R. 

(3)  Sample  units  shali  be  chosen  with  frequency  (f)  so  as  to  give  each  unit  of  product  an  equal  chance  of 
being  inspected.  The  inspector  should  allow  the  interval  between  samf^e  units  to  vary  somewhat 
rather  than  draw  sample  units  according  to  a  rigid  pattern. 


4.9.3. 1  Conditions  for  Continuous  Sampling  Procedures 

The  following  conditions  must  exist  before  the  continuous  attributes  sampling 
procedures  of  this  section  may  be  used  for  inspection. 

•  Moving  product. 

•  Ample  space,  equipment,  and  manpower  at  or  near  the  inspection  station  to 
permit  100  percent  inspection  when  required. 

•  A  process  that  is  producing  or  is  capable  of  producing  material  whose  quality 
is  stable. 
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4. 9. 3. 2  Continuous  Sampling  Inspection  Procedure 

At  the  start  of  production,  all  units  are  inspected.  Sampling  inspection  may  be 
initiated  at  frequency/when  the  following  condidons  are  satisfied: 

(a)  All  units  of  product  are  of  the  same  configuration  and  produced  under  stable 
conditions. 

(b)  At  least  i  consecutive  units  inspeaed  are  free  of  nonconformances. 

Sampling  inspection  shall  be  terminated  and  100  percent  inspection  resumed  if 
either  or  both  of  the  following  condititms  occur. 

(1)  The  production  process  is  interrupted  for  more  than  three  operating  days,  or 
the  requirement  of  (a)  above  is  otherwise  not  satisfied. 

(2)  A  unit  having  any  nonconformance  is  found  during  sampling. 
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5.  ALTERNATE  ACCEPTANCE  PROVISIONS 


5.1  GENERAL 

This  standard,  when  referenced  in  the  contract  or  product  specifications,  requires 
the  contractor  to  perform  sampling  inspection  in  accordance  with  Section  4  and  the  product 
specification.  However,  it  is  recognized  that  sampling  inspection  alone  does  not  control  or 
improve  quality.  Product  quality  comes  from  proper  product  and  process  design  and 
process  control  activities  and  when  they  are  effective,  sampling  inspection  is  a  redundant 
effort  and  an  unnecessary  cost.  Contractors  that  have  an  acceptable  quality  system  and 
proven  process  controls  on  specitic  processes  are  encouraged  to  consider  submitting 
alternate  acceptance  methods  for  one  or  more  contractually  specified  characteristic.  In 
addition,  contractors  that  have  a  successful  quality  system  and  a  history  of  successful 
process  controls  throughout  the  company  are  encouraged  to  consider  submitting  a  systemic 
alternate  acceptance  method  for  all  the  contractual  sampling  inspection  requirements 
associated  with  Section  4. 

Submissions  will  describe  the  alternate  acceptance  methods,  the  sampling 
inspection  provision  to  be  replaced,  and  an  evaluation  of  the  protection  provided  by  the 
alternate  methods  as  compared  with  the  inspection  requirement  to  be  replaced.  The 
alternate  acceptance  method  shall  include  evidence  of  process  control  and  capability  during 
production  together  with  adequate  criteria,  measurement,  and  evaluation  procedures  to 
maintain  control  of  the  process.  The  acceptability  of  the  alternate  acceptance  methods  is 
dependent  on  the  existence  of  a  quality  system,  the  demonstration  of  its  process  focus,  and 
the  availability  of  objective  evidence  of  effectiveness. 

5.2  REQUIREMENTS  AND  PROCEDURES 

Contractors  currently  operating  quality  systems  in  accordance  with  such  models  as 
MIL-Q-9858  enhanced  with  Statistical  Process  Controls  (SPC),  ANSI/ASQC  Q94,  or 
others  that  are  deemed  satisfactory  to  the  government  representative  are  qualified  to  apply 
for  alternate  acceptance  methods  if  demonstration  of  process  focus  and  objective  evidence 
of  effectiveness  exists. 
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The  contractor  will  include  in  his  request  for  alternate  acceptance  method  approval 
an  assessment  plan  to  periodically  verify  process  stability,  capability,  and  other  conditions 
under  which  the  alternate  acceptance  method  was  developed.  The  current  target  values  of 
process  capability  are  equivalent  to  a  Cpk  of  2.00  for  critical  characteristics,  1.33  for  major 
characteristics,  and  1.0  for  minor  characteristics.  Upon  approval  of  the  assessment  plan, 
the  contractor  may  reduce  or  eliminate  inspection  sampling  when  the  plan  criteria  are  met  or 
exceeded. 

Appendix  B  provides  criteria  and  considerations  that  may  be  used  if  the  contract 
does  not  otherwise  establish  procedures  for  alternate  acceptance  methods. 

5.3  SUBMISSION  AND  INCORPORATION 

There  are  two  ways  of  sulxnitting  alternate  acceptance  methods: 

1.  Submission  of  individual  alternate  acceptance  methods  for  one  or  more 
contractually  specified  sampling  inspection  requirements  to  the  Government 
quality  assurance  representative  (QAR)  for  approval  at  any  time  during  the 
contract  period  of  performance. 

2.  Submission  of  a  systemic  alternate  acceptance  method  to  the  procuring 
contracting  officer  (PCO)  prior  to  contract  being  awarded.  This  pre-approval 
allows  the  contractor  to  adopt  alternate  acceptance  methods  throughout  the 
length  of  the  contract.  After  contract  award,  submissions  of  a  systemic 
alternate  acceptance  method  should  be  made  through  the  administrative 
contracting  officer  (ACX))  to  the  PCX). 

All  approved  alternate  acceptance  methods  shall  be  incorporated  into  the 
contractor’s  manufacturing  and  quality  program  plans  or  other  vehicles  acceptable  to  the 
ccmtracting  agency,  as  applicable. 

5.4  WITHDRAWAL  OF  APPROVAL  OF  ALTERNATES 

The  government  reserves  the  right  to  withdraw  approval  of  alternate  acceptance 
methods  that  are  determined  to  provide  less  assurance  of  quality  than  the  inspection 
requirements  originally  specified  or  when  the  inability  to  maintain  process  stability  and 
c^ability  over  time  becomes  q)paient 
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6.  NOTES 


6.1  REFERENCES 

6.1.1  Government  Documents 

•  MIL-STD-105E,  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes. 

•  MIL>STD-414,  Sampling  Procedures  and  Tables  for  Inspection  by  Variables 
for  Percent  D^ective. 

•  MIL-STD-123SC,  Single  and  Multi-level  Continuous  Sampling  Procedures 
and  Tables  for  Inspection  by  Attributes. 

•  DoD  4245.7,  Transition  from  Development  to  Production. 

•  MIL-Q-98S8  A,  Quality  Program  Requirements. 

•  MIL-I-45208A,  Inspection  System  Requirements. 

6.1.2  Commercial  Documents 

•  ANSI  Zl.l/ASQCBl,  Guide  for  QueUityConttol  Charts 

•  ANSI  ZL2/ASQC  B2,  Control  Chart  Methods  of  Analyzing  Data 

•  ANSI  Z1 .3/ASQC  B2,  Control  Chart  Method  of  Controlling  Quality  During 
Production 

•  ANSI/ASQC  Q90,  Quality  Management  and  Quality  Assurance  Standards — 

Guidelines  for  Selection  Use. 

•  ANSI/ASQC  Q94,  Quality  Management  and  Quality  System  Elements — 
Guidelines. 

•  ISO  9000,  Quality  Management  and  Quality  Assurance  Standards — Guidelines 
for  Selection  and  Use. 

•  ISO  9004,  Quality  Management  and  Quality  System  Elements— Guidelines. 
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Annex  A 

EXAMPLES  OF  SAMPLING  PLAN  USE 


1 .  Attributes  Sampling 

Wing  nuts  are  to  be  inspected  for  missing  thread.  A  verification  level  IV  (VL-IV) 
has  been  specified.  The  producer  chooses  to  use  attributes  sampling  plans  from  Table  2. 
Lot  sizes  may  vary  as  a  result  of  production  decisions.  A  segment  of  the  producer’s 
experience  is  shown  in  Figure  A-1. 


Lot# 

Lot 

Size 

Code 

Letter 

Sample 

Size 

Non¬ 

conform¬ 

ances 

Lot 

Disposition 

Stage 

T/N/R 

Action 

1 

5000 

■ 

160 

2 

Withhold 

Acceptance 

N 

Begin  with  normal 
sampling,  VL-IV. 

2 

900 

80 

Accept 

N 

3 

3000 

C 

128 

1 

Withhold 

Accpotance 

N 

2  lots  out  of  5  fail  to  pass. 
Switch  to  tightened  VL-IV. 

4 

1000 

B 

256 

0 

Accept 

T 

5 

B 

256 

0 

Accept 

T 

6 

A 

192 

0 

Accept 

T 

7 

C 

0 

Accept 

T 

8 

C 

0 

Accept 

T 

9 

3000 

C 

128 

0 

Accept 

N 

5  consecutive  lots 
accepted.  Process 
corrected.  Switch  to 
normal  VL-IV. 

10 

5000 

D 

160 

0 

Accept 

N 

Figure  A>1.  Attributes  Sampling  Inspection  Log 
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2.  Variables  Sampling  (Single  Specification  Limit  Case) 

The  maximum  temperature  of  operation  for  a  certain  device  is  specified  as  209 
(measured  in  degrees  F).  Verification  level  I  (VL-I)  has  been  specified.  A  lot  of  40  items 
is  submitted  for  inspection  in  accordance  with  variables  sampling.  Table  3  requires  a 
sample  size  of  nv  =  4  for  code  letter  A  (CL- A).  Suppose  the  measurements  obtained  are  as 
follows:  197,  188,  184,  and  205;  and  compliance  with  the  acceptability  criteria  is  to  be 
determined.  Cmnputations  are  shown  in  Figure  A-2. 


Line 

Information  Needed 

Symbol 

Formula 

Result 

Explanation 

1 

Sample  size 

nv 

4 

See  Table  3 

Sum  of  measurements 

2X 

774 

Sum  of  squared  measurements 

ZX2 

150034 

B 

Correction  factor 

CF 

(IX)2/nv 

149769 

(774)2/4 

H 

Corrected  sum  of  squares 

SS 

£X2.CF 

265 

150034-149769 

H 

Sample  variance 

V 

SS/(nv-1) 

88.333 

265/3 

B 

Sample  standard  deviation 

Vv 

9.399 

V88.333 

H 

Sample  mean 

£X/ny 

193.500 

774/4 

H 

Lower  specification  limit 

Not  applicable 

■ 

Upper  specification  limit 

U 

209 

10 

Lower  quality  index 

Ql 

{X-L)/S 

Not  applicable 

Upper  quality  index 

Qu 

(U-X)/s 

1.649 

(209-1 93.5)/9.399 

Quality  Index  «  Min(QL,  Qu) 

■■ 

min{QLQu) 

1.649 

11 

Sample  F  value 

Hi 

s/(U-L) 

Not  applicable 

12 

Number  of  nonconformances 

c 

0 

k  value 

k 

1.210 

See  Table  3 

F  value 

F 

Not  applicable 

See  Table  3 

13 

C  acceptability  criterion 

C»0? 

Yes 

k  acceptability  criterion 

Q^k? 

Yes 

1.649  ^  1.21 

F  acceptability  criterion 

F  ^F? 

Not  applicable 

NOTES:  The  k  value  is  the  minimum  albwabie  value  for  the  quality  index,  Q. 

The  F  value  is  the  maximum  alloweMe  value  for  the  sample  F  value, 

F. 

Figure  A-2.  Computations  for  Single  Specification  Limit  Case 

The  lot  is  accepted  because  it  meets  all  applicable  acceptability  criteria. 
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3.  Variables  Sampling  (Double  Specification  Limit  Case) 


The  minimum  temperature  of  operation  for  a  certain  device  is  specified  as  180 
(measured  in  degrees  F).  The  maximum  is  209.  Verification  level  I  (VL-I)  has  been 
speciHed.  A  lot  of  40  items  is  submitted  for  inspection  in  accordance  with  variables 
sampling.  Table  3  requires  a  sample  of  size  nv  =  4  for  code  letter  A  (CL-A).  Suppose  the 
measurements  obtained  are  as  follows:  197,  188,  184  and  205;  and  compliance  with  the 
acceptability  criteria  is  to  be  determined.  Computations  are  shown  in  Figure  A-3. 


Line 

Information  Needed 

Symbol 

Formula 

Result 

Explanation 

1 

Sample  size 

nv 

4 

See  Table  3 

2 

Sum  of  measurements 

IX 

774 

3 

Sum  of  squared  measurements 

1X2 

150034 

n 

Correction  factor 

CF 

(lX)2/nv 

149769 

(774)2/4 

H 

Corrected  sum  of  squares 

SS 

2x2^F 

265 

150034-149769 

H 

Sample  variance 

V 

SS/(nv-1) 

88.333 

265/3 

7 

Sample  standard  deviation 

s 

Vv 

9.399 

V88.333 

8 

Sample  mean 

X 

iX/nv 

193.500 

774/4 

Lower  specification  limit 

L 

180 

Upper  specification  limit 

U 

209 

10 

Lower  quality  index 

Ql 

(X-L)/s 

1.436 

(1 93.5-1 80)/9.399 

Upper  quality  index 

Qu 

(U-X)/s 

1.649 

(209-193.5)/9.399 

Quality  Index  -  Min(QL,  Qu) 

Q 

1.436 

11 

Sample  F  value 

F 

s/{U-L) 

0.324 

9.399/(209-180) 

12 

Number  of  nonconformances 

C 

k  value 

k 

See  Table  3 

F  value 

F 

See  Table  3 

13 

C  acceptability  criteria 

Yes 

k  acceptability  criteria 

Yes 

1.436  ^1.210 

F  acceptability  criteria 

wbm 

Yes 

0.324  S  0.370 

NOTES:  The  k  value  is  the  minimum  allowabie  value  for  the  quality  index,  Q. 

The  F  value  is  the  maximum  allowabie  value  for  the  sanple  F  value,  F . 

Figure  A>3.  Computations  for  Double  Specification  Limit  Case 

The  lot  is  accepted  because  it  meets  all  applicable  acceptability  criteria. 
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4 .  Continuous  Sampling 


A  visual  inspection  of  stamped  metal  parts  for  the  presence  of  a  spot  weld  will  be 
performed  immediately  after  units  pass  through  a  spot  welding  station.  Verincation  level  n 
(VL-II)  has  been  specified.  The  product  will  be  submitted  for  continuous  attributes 
sampling  inspection.  The  production  interval  size  is  an  8-hour  shift,  which  initially  will 
consist  of  between  700  to  800  welded  parts.  With  VL-II  and  code  letter  C  (CL-C)  from 
Table  1,  the  i  and/ values  (Table  4)  are  found  to  be  1 16  and  1/48,  respectively.  A  segment 
of  sampling  experience  is  shown  in  Hgure  A-4. 


Product 

Item 

Number 

Code 

Letter 

Frequency 
or  100% 

Stage 

T/N/R 

Event/Action 

1 

C 

100% 

N 

Start  production:  Begin  screening  phase  with  /  -  1 16. 

8 

C 

100% 

N 

Find  a  defective  unit:  Reset  counter. 

124 

C 

100% 

N 

/  «1 16  consecutive  conforming  units  cleared:  Begin 
sampling  phase  with  /-  1/48. 

C 

N 

First  random  sampie  selected:  Found  it  to  conform. 

C 

N 

200  consecutive  conforming  sampied  units  obsenred: 
Switch  to  reduced  inspection  with  /«  1/68.  Here,  200 
equals  10  times  the  Table  2  sarrple  size  entry  for  CL-C 
and  VL-II. 

9769 

C 

R 

Next  sample  randomly  selected  with  /« 1/68. 

13982 

C 

H 

R 

Production  intenral  size  tripled  (2100  to  2400  units): 

End  CL-C  and  begin  CL-E  sampling  phase,  /«  1/136, 
since  VL-II  and  reduced  sampling  inspection  are  in 
effect. 

14121 

E 

1/136 

H 

First  random  sample  taken  with  new  f  =  1/136:  Found  it 
to  conform.  Continue  random  sampling. 

16290 

E 

1/136 

H 

A  nonconforming  unit  observed:  Switch  to  normal 
inspection.  Initiate  screening  phase  with  i  >  228,  since 
CL-E  and  VL-II  are  in  effect. 

16518 

E 

100% 

N 

/  B  228  consecutive  conforming  units  cleared:  Begin 
sampling  phase  with  /-  1/96. 

Figure  A>4. 


Continuous  Sampling  Inspection  Log 
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5 .  Continuous  Sampling  (Producer  Alternate) 

The  producer  may  opt  to  use  another  continuous  sampling  plan  instead  of  the  one 
specified  in  Table  4.  The  only  restrictions  are  that  such  a  change  is  not  allowed  while 
inside  a  screening  sequence  and  that  the  new  plan  be  derived  in  accordance  with  the 
procedure  described  below. 

Certain  circumstances  make  such  choices  desirable.  Sometimes  the  selection  of  a 
clearance  number  or  frequency  is  application  dependent,  e.g.,  if  it  matters  that  i  or  l/f  be  a 
multiple  of  pallet  size.  Availability  and  capability  of  screening  and  sampling  crews  are  yet 
further  considerations. 

The  plan  cited  in  Table  4  consists  of  the  largest  /  number  and  the  smallest /  number 
combination.  Plans  whose  i  is  larger  than  the  tabulated  i,  or  whose/  is  smaller  than  the 
tabulated/,  are  not  permitted.  Producers  willing  to  sample  at  rates  larger  than/  can  reduce 
i  substantially. 

The  procedure  that  allows  choice  is  presented  by  way  of  the  preceding  continuous 
sanq)ling  example  situation  as  initially  described,  subject  to  one  modincaticm;  the  producer 
prefers  to  start  with  a  plan  having  an  i  of  50  instead  of  the  1 16  specified.  The  procedure  to 
determine  a  valid/  is  as  shown  in  Rgure  A-5. 


Line 

Information  Needed 

Symbol 

Formula 

Result 

Explanation 

1 

Clearance  number 

■■ 

116 

Table  4 

Target  i  number 

BH 

lt<i? 

Yes 

50  <  116 

Attribute  Sample  Size 

20 

Table  2,  same  VL,  CL 

B 

Compute  fo; 

B 

Step  1 

S1 

(na+1)(1+1/na)'’a 

55.7193 

Step  2 

(l,+1)(1+1/-|t)H 

137.2710 

steps 

(S1/(S1-1)JH 

2.4732 

B 

Step  4 

to 

(S1-1)/[(S2){S3)] 

0.1612 

D 

Valid  f 

Anyf  >fo 

1/6 

1/6  >0.1612 

Figure  A-5.  Procedure  to  Determine  a  Vaiid  f 

Therefore,  an  i  of  SO  may  be  used  in  lieu  of  1 16  if /is  increased  from  1/48  to  1/6. 

If  it  is /that  is  preselected,  the  corresponding  /  may  be  found  by  trial  and  error,  that 
is,  by  iterative  implementation  of  the  procedure  described. 
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The  printed  numerical  results  have  been  rounded  to  4-decimal  accuracy.  However, 
use  of  the  procedure  requires  that  all  calculations  be  performed  with  at  least  6-digit 
precision.  Evidence  supporting  the  validity  of  numerical  results  shall  be  maintained  and  be 
available  for  review  upon  request.  Proper  execution  of  the  procedure  ensures  Tables  4  and 
2  are  comparable  with  respect  to  the  average  fraction  inspected  and  the  average  outgoing 
quality  limit. 
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Annex  B 

ALTERNATE  ACCEPTANCE  PROVISIONS— 
REQUIREMENTS  AND  PROCEDURES 


1 .  QUALITY  SYSTEM 

In  order  for  an  alternate  acceptance  n^thod  to  be  considered,  the  contractor  shall 
establish  and  utilize  an  internal  quality  system  as  a  means  of  ensuring  that  all  products 
conform  to  requirements  specified  by  the  contract  and  associated  specifications  and 
standards.  The  quality  system  shall  be  documented  and  shall  be  subject  to  on-site 
government  review  throughout  the  contract.  It  shall  include,  at  a  minimum,  a  description 
of  the  organizational  structure,  responsibilities,  procedures,  processes,  and  resources. 
Such  documentation  is  hereinafter  called  the  quality  system  plan.  The  contractor  shall 
maintain,  disseminate,  update,  and  improve  the  quality  system  plan  in  order  to  ensure  its 
ccxitinued  use  and  accuracy.  The  design  and  documentation  of  the  quality  system  plan  shall 
allow  for  ease  of  use,  review,  and  audit  by  internal  as  well  as  government  personnel. 

The  quality  system  shall  be  prevention-based.  Common  quality  system  models  that 
reflect  this  philosophy  include  the  ISO  9000  series,  MIL-Q-9858  enhanced  with  SPC,  and 
many  industry  specific  total  quality  standards  and  programs.  The  quality  system  shall  also 
reflect  additional  needs  in  accordance  with  the  requirements  of  this  standard.  Regardless  of 
the  model  chosen,  the  quality  system  shall  demonstrate  its  effectiveness  by  meeting  the 
following  objectives  throughout  all  areas  of  contract  performance: 

•  The  quality  system  is  understood  and  executed  by  all  personnel  having  any 
influence  on  product  or  process  quality. 

•  Products  and  services  meet  or  exceed  customer  requirements. 

•  Quality  is  deliberately  and  econcxnically  controlled. 

•  Emphasis  is  on  the  prevention  of  process  discrepancies  and  product 
nonctmfcnmances. 

•  Discrepancies  and  nonconformances  that  do  occur  are  readily  detected,  and 
root  cause  corrective  actions  are  taken  and  verified. 
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•  Sound  problem  solving  and  statistical  methods  are  employed  to  continuously 
reduce  process  variability  and,  in  tum,  improve  process  capability  and  product 
quality. 

•  Records  are  maintained  and  indicate  in^lementation  process  of  the  quality  plan 
and  effectiveness  of  the  control  procedures. 

The  acceptability  of  the  quality  system  as  part  of  the  request  for  alternate  acceptance 
method(s)  is  dependent  on  its  compliance  with  an  industry  accepted  quality  system  model, 
demonstration  of  its  process  focus,  and  the  availability  of  objective  evidence  of  its 
effectiveness  as  described  below. 

2 .  PROCESS  FOCUS  OF  QUALITY  SYSTEM 

To  demonstrate  a  process  focus,  the  contractor  shall  show  that  the  manufacturing 
process  and  its  related  processes  have  been  studied  and  are  understood,  controlled,  and 
documented  in  such  a  manner  that  they  are 

•  Consistently  producing  conforming  product 

•  Controlled  as  far  upstream  as  possible. 

•  Robust  to  variation  in  equipment  raw  materials,  and  other  process  inputs,  and 
designed  to  yield  a  quality  product. 

•  Operated  with  the  intent  to  constantly  strive  to  reduce  process/product 
variability. 

•  Designing  or  procuring  manufacturing  equipment  with  objectives  of  minimum 
variability  around  targeted  values. 

•  Managed  for  continuous  improvement 

•  Designed  and  controlled  using  a  combination  of  manufacturing  practices  and 
statistical  methods  in  order  to  ensure  defect  prevention  and  process 
improvement 

3 .  OBJECTIVE  EVIDENCE  OF  QUALITY  SYSTEM  IMPLEMENTATION 
AND  EFFECTIVENESS 

3.1  Examples  of  Evidence  Regarding  Process  Improvement 

•  Process  flow  charts  showing  the  key  control  points  where  action  is  taken  to 
prevent  the  production  of  defective  product. 
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•  Identification  of  process  improvement  techniques  and  tools  used,  e.g.,  Plan- 
Do-Check-Act  (PDCA)  cycle.  Failure  Modes  and  Effects  Analysis  (FMEA), 
Pareto  Analysis,  and  Cause  and  Effect  Analysis. 

«  Identiffcation  of  the  measures  used,  e.g.,  trend  analysis,  cost  of  quality,  cycle 
time  reduction,  defect  rates,  6  sigma  capability. 

•  Results  of  the  improvements  from  the  use  of  these  process  in^rovement  tools. 

•  Results  of  properly  planned  experiments  that  led  to  reduced  common  cause 
variability  of  a  process  and  improved  productivity. 

3.2  Examples  of  Evidence  Regarding  Process  Control 

•  Identification  of  the  scope  of  use  of  process  control  techniques,  e.g.,  SPC, 
automation,  gages,  set-up  verification,  preventative  maintenance,  visual 
inspection. 

•  Process  control  plans,  including  the  improvement  goals  and  statements  of 
management  commitment  to  SPC. 

•  Approaches  and  supporting  data  used  to  determine  if  suppliers  have  adequate 
controls  to  assure  defective  product  is  not  produced  and  delivered. 

•  Descriptions  of  the  required  training  in  SPC  and/or  continuous  improvement, 
i.e.,  the  number  of  courses,  their  content,  courses  required  for  personnel  at 
each  organizational  level  and  function  associated  with  the  quality  plan,  the 
qualiffcations  of  the  instructors  or  trainers  for  SPC  classes,  support  by 
management  to  attend  such  courses,  and  information  demonstrating  the 
effectiveness  of  the  training. 

•  Identiffcation  and  definition  of  the  interrelations  of  all  departments  (e.g., 
production,  engineering,  purchasing,  marketing,  administration,  etc.)  involved 
in  SPC  and  quality  improvement,  their  responsibilities,  and  the  use  of  teams. 

•  When  applying  control  charts,  the  basis  for  criteria  on  establishing  rational 
subgroups,  the  frequency  of  sampling,  and  the  proper  procedures  for 
establishing  and  updating  control  limits. 

•  Identiffcation  of  key  parameters,  placement  of  their  control  points  in  the 
manufacturing  process,  and  method  of  verifying  the  correlation  of  such 
parameters  when  they  are  used  in  lieu  of  one  ot  more  specified  characteristics. 

•  Basis  for  criteria  on  determining  out-of-control  conditions,  and  identiffcation  of 
personnel  responsible  for  process-related  corrective  action. 

•  Proper  gage  measurement  studies  showing  measurement  variations  relative  to 
the  total  variatitm. 
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•  Traceability  of  the  product  and  process  corrective  action(s)  taken  when  the 
process  went  out  of  statistical  control,  showing  how  the  root  cause  was 
identified  and  eliminated. 

3.3  Examples  of  Evidence  Regarding  Product  Conformance 

•  Ctxitrol  charts  shoudng  the  process  to  be  in  statistical  control. 

•  Records  of  product  and  process  corrective  action(s)  taken  when 
nonconformances  occur. 

•  Process  capability  studies  consisting  of  the  correct  calculation  and 
interpretation  of  indices,  such  as  Cp  and  Cpk- 

•  History  of  product  inspection  results  reinforced  by  statistical  data  and  analysis. 

•  Results  from  in-process  control  methods,  such  as  100  percent  automated 
assembly  and/or  inspection. 
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parts  used  in  its  equipment.  Considerable  investment  was  required  to 
manage  the  stock  of  a  continually  growing  parts  inventexy.  The  company's 
standards  engineering  department  was  autiiorized  to  investigate  and  propose 
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E-1 


American  Defense  Preparedness  Association  (ADPA),  Statistical  Process  Control  (SPC) 
Symposium,  “SPC:  A  Management  Strategy  for  Survival,”  Proceedings,  San  Diego,  CA, 
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Internal  customers  Perception  of  TQM  (Lawrence  J.  Schrader,  CONCO, 

Inc.). 

On  Moving  to  Operator  Self-Assessment  (Dr.  Ken  Tieman,  Loral 
Aeronutronic). 

SPC:  The  Next  Generation  (Anand  M.  Joglekar,  Alliant  Techsystems, 

Inc.). 

SPC  Implementation:  Management  Issues  ((Tathy  Lawson,  Motorola). 

SPC  Control  at  General  Dynamics  Land  Systems  (Michael  A,  Puzzuoli, 
General  Dynamics  Land  Systems). 

Administrative  SPC  and  Measurement  (Alexander  E.  Lewis,  Mason  & 

Hanger  Silas  Mason,  Co.). 

The  Customer’s  Role  in  Supplier  SPC  Efforts  (Thomas  O.  Moore, 
COMARCO). 

SPC:  A  Management  Strategy  for  Survival  (Jennifer  Kibiger,  Olin 
Ordnance). 

Statistical  Process  Control  in  the  Hybrid  Microelectronics  Manufacturing 
Industry;  a  Navy  View  Point  (Charles  C.  Azu,  Jr.,  Naval  CC&OS  Center). 

Profiting  from  the  Integration  of  SPC  and  S(3C  (Roy  Aliminas,  Palmer 
Manufacturing  C^.). 

Paint  Thickness  Process  Improvements  (Lisa  A.  Steele,  Lockheed 
Aeronautical  Systems  Co.). 

Statistical  Design  of  Experiments  for  an  Improvement  in  Ml 36  (AT4) 
Performance  (Wayne  W.  Corrigan,  Alliant  Techsystems,  Inc.). 

Key  Characteristic  Identification  (Robyn  Rosser,  Vought  Aircraft  Co.). 

SPC  Implementation  via  a  TQM  Team  Structure  (Dana  Cook,  Olin 
Ordnance). 

SPC  and  the  Bottom  line  (Jim  Woolnough,  Loral  Aeronutronic). 

Managing  Ship  Performance  of  Naval  Gunfire  Support  Using  Statistical 
Process  Control  (M.  Bailey,  Naval  Postgraduate  School;  J.  Bowde,  Naval 
Surface  Warfare  Center,  and  Alexander  C^ahan,  Sr.,  COMARCO). 

SPC  With  a  Twist  (Richard  Tucker,  Numerical  Control  Support,  Inc.). 

Total  Improvement  Management  (Dr.  H.  James  Harrington,  Ernst  & 
Young). 
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IQUE  and  Contractor  Performance  Improvement  (Col.  Tom  Barnes, 
DCMC,  Fort  Belvoir,  and  Bernard  P.  (Carroll,  Loral  Aeronutronic). 

Lessons  Learned  in  the  Application  of  SPC  to  the  R&D  Process  at  Hamilton 
Standard  (Daniel  Kelly,  Hamilton  Standard  Division  of  United 
Technologies). 

Identification  of  SPC  Process  Control  Points  with  Failure  Modes  and 
Effects  Analysis  (FMEA)  (Julio  J.  Rivera,  Quantic  Industries). 

American  Defense  Preparedness  Association  (ADPA),  Statistical  Process  Control  (SPC) 
Tutorial,  Track  2,  “Advanced  SPC,”  16  March  1993. 

American  Defense  Preparedness  Association  (ADPA),  Statistical  Process  Control  (SPC) 
Tutorial,  Track  3,  “Qutdity:  A  Management  Strategy  for  Profitability,”  16  March  1993. 

American  Defense  Preparedness  Association,  Second  Annual  Statistical  Process  Control, 
SPC  ^plication  and  Integration,  Proceedings,  Hunt  Valley,  MD,  1-4  March  1^2. 

The  Shewhart  Lectures  of  1938 — ^Profound  Knowledge  at  Its  Best,  John  H. 
Peterson 

Statistical  Process  Control  as  a  Contractual  Quality  Requirement  for 
Conventional  Ammunition,  John  Bowden  and  Robert  Chvatal 

ABC:  A  Sanqiling  Plan  to  Attain  Process  (Control  and  Capability,  (Tarmen  J. 

Liuzza,  Paul  A.  Roediger,  and  Edward  G.  Schilling 

Pre-Control  vs.  Control  Charting:  A  Statistical  Comparison,  Neal  A. 
Macheitich 

Improving  "White  Collar"  Quality  Through  Process  Assessment,  Ken 
Tieman  and  Denis  Faunce 

SPC  Implementation  at  the  Naval  Ordnance  Detachment,  Yorktown,  Lester 
B.  Leonard,  m 

SPC  Training:  Doing  it  Right  the  First  Hme,  Robin  McDermott 

SPC  a  Success  Stoiy  at  Conco,  Inc.,  Lawrence  J.  Schrader 

Integration  of  SPC  with  the  Life  Cycle  System  Management  Model,  Ray 
Hamblin 

Low-Volume  SPC  Application  Used  in  Problem  Diagnosis  and  Vendor 
Evaluation,  Phillip  E.  Dwker 

Printed  Circuit  Card  Assembly  Process  Variability  Reduction  Through 
SPC,  Mariana  Purer 

The  Critical  Relationship  of  Total  Quality  Management  to  SPC,  John  J. 
Grunwald  and  James  A.  Dickerson 

Application  of  the  Outlinier  Test  of  SPC  to  Quality  Acceptance  Procedures, 

Louie  J.  Lipp 

SPC — ^The  Two  Edged  Panacea,  Alben  M.  Levenson 

American  Defense  Preparedness  Association  (ADPA),  “In-Plant  Quality  Evaluation 
(IQUE)/Ccmtractor  Performance  Briefing,”  Defense  Contract  Management  Office  (DCMO), 
viewgraphs,  Loral,  Newport  Beach,  17  March  1993. 

American  National  Standards  Institute  (ANSI) — ^Brochures. 


E-3 


American  Society  for  Quality  Control,  ANSI  ASC  Z-1  Committee  on  Quality  Assurance 
Answers  the  Most  Frequently  Asked  Questions  About  the  ISO  9000  (ANSIIASQC  Q90) 
Series,  pamphlet,  Milwaukee,  WI. 

American  Society  for  Quality  Control  (ASQC),  Quality... It's  Everybody's  Journey, 
pamphlet,  Milwaukee,  WI. 

American  Society  for  Testing  and  Materials  (ASTM),  What  is  ASTM?,  pamphlet, 
Philadelphia,  PA. 

American  Welding  Society,  Industry-Government  Standards  Partnership  IV,  Conference 
Summary,  Williamsburg,  VA,  20-22  November  1991. 

Ashley,  Steven,  “Applying  Taguchi’s  Quality  Engineering  to  Technology  Development,” 
Mectwiical  Engineering,  July  1992,  pp.  58-60. 

Now  that  General  Taguchi’s  quality-engineering  techniques  are  starting  to 
find  increased  acceptance  among  U.S.  manufacturing  firms,  the  Japanese 
quality  guru  is  a^ocating  the  applicatitm  of  his  design-optimization  system 
earlier  in  the  new  product  cycle — to  the  technology-development  stage, 
where  the  payoffs  are  even  greater. 

Beilinson,  H.A.,  "Early  Difficulties  in  the  Way  of  a  Formalized  Allowance  for  a  Percentage 
of  Defectives,"  American  Statistical  Association,  Washington,  DC,  1950. 

Comments  taken  from  "Acceptance  Sampling — A  Symposium,"  pages 
46-47,  part  of  a  prepared  discussion  of  two  papers  on  "Acceptance 
Sampling  by  Attributes,"  by  Paul  Peach  and  E.G.  Olds.  The  symposium 
was  held  at  the  Erst  postwar  meeting  of  the  American  Statistical  Association 
at  Cleveland,  OH,  27  January  1946. 

"AOQL:  A  Quick  Reference,"  Quality,  July  1991,  page  Q-32. 

Boyd,  Bill,  Rockwell  Collins,  "ADPA  SPC  Committee  Definitions  of  CPI  and  SPC," 
3  March  1992. 

Breitenberg,  Maureen,  “Questitms  and  Answers  on  Quality,  the  ISO  9000  Standard  Series, 
Quality  System  Registration,  and  Related  Issues,”  U.S.  Department  of  Commerce, 
National  Institute  of  Standards  and  Technology,  Standards  Code  and  Information  Program, 
Gaithersburg,  MD,  Revised  July  1992. 

This  report  provides  information  on  the  development,  content  and 
application  of  the  ISO  9000  standards  to  readers  who  are  unfamiliar  with 
these  aspects  of  the  standards.  It  attempts  to  answer  some  of  the  most 
commonly  asked  questions  on  quality;  quality  systems;  the  content, 
application  and  revision  of  the  ISO  9000  standards;  quality  system 
approval/registration;  European  Community  requirements  for  quality  system 
approval^egistration;  and  sources  for  additional  help. 

Brodsky,  Jeffrey,  Philip  Crosby  Associates,  Inc.,  "Zero  Defect  Supplier  Selection," 
Quality,  June  1991,  p.  18. 

"The  Cracks  in  Quality,"  The  Economist,  18  April  1992,  pp.  67-68. 

Many  western  managers  believe  that  total-quality  management  is  a  powerful 
weapon,  especially  against  the  Japanese.  The  trouble  is,  most  of  their 
quality  programs  are  not  delivering  the  goods. 
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Chan,  Lai  K.,  Smiley  W.  Cheng,  and  Frederick  A.  Spiring  (University  of  Manitoba, 
Canada),  “A  New  Measure  of  Process  Capability:  Cpm,”  Journal  of  Quality  Technology, 
Volume  20,  Number  3,  July  1988,  pp.  162-175. 

A  new  measure  of  the  process  capability  (Cpm)  is  proposed  that  takes  into 
account  the  proximity  to  the  target  value  as  well  as  ^e  process  variation 
when  assessing  process  performance.  The  sampling  distribution  for  an 
estimate  of  Cpm  (Cpm)  and  some  of  its  properties  have  been  examined  and 
an  example  of  its  applications  is  included.  The  new  index  is  easy  to 
compute  and,  with  the  aid  of  the  included  tables,  easy  to  analyze.  Cpm  has 
S(xne  mcne  desirable  statistical  properties  than  Cp  anti  CpL  the  estimates  of 
the  Cp  and  Cpk  indices,  respectively. 

Cheng,  Smiley  W.  (National  Sciences  and  Engineering  Research  Council  of  Canada)  and 
Frederick  A.  Spiring  (University  of  Manitoba,  Canada),  “Assessing  Process  Capability:  A 
Bayesian  Approach,”  Industrial  Engineering  Research  and  Development,  Volume  21, 
Number  1,  March  1989,  pp.  97-98. 

Quality  Control  Practitioners  often  base  inferences  regarding  the  capability 
of  a  process  on  a  point  estimate  without  examining  the  distributionid 
qualities  of  the  estimator  used.  A  Bayes  solution  is  proposed  that  provides 
good  statistical  analysis  that  can  be  easily  used  and  interpreted  on  the 
manufacturing  floor. 

Chou,  Youn-Min  (University  of  Texas,  San  Antonio),  D.B.  Owen  (Southern  Methodist 
University,  Dallas),  and  Salavador  A.  Borrego  (Monterrey,  Mexico),  “Lower  C^onfidence 
Limits  on  Process  Capability  Indices,”  Journal  of  Quality  Technology,  Volume  22, 
Number  3,  July  1990,  pp.  223-229. 

Lower  confidence  limits  are  derived  for  the  common  measures  of  process 
capability,  usually  indicated  by  Cp,  CPU,  CPL,  and  Cpk.  The  measures  are 
estimated  based  on  a  random  sample  of  observations  from  the  process  when 
the  process  is  assumed  to  be  normally  distributed  and  has  reached  a  state  of 
statistical  control. 

Deutch,  John,  Under  Secretary  of  Defense  (Acquisition  and  Technology),  News  Release 
on  Authorization  of  ISO  9(X)0. 

Fain,  LTG  James  A.  Jr.,  ASC  Best  Practice  on  Advanced  Quality  Requirements  for 
Acquisition  Programs,  Commander,  Department  of  Air  Force  ASC7CC,  Wright-Patterson 
Air  Force  Base,  OH,  20  November  1993. 

Ford  Motor  Ckimpany,  Miscellaneous  Pamphlets:  Quality,  Mission  Values,  Letter  reply  to 
Van  Atta. 

Freund,  Richard  A.(Quality  Planning  Services,  Rochester,  NY)  and  Edward  G.  Schilling 
(Rochester  Institute  of  Technology),  "Standards  in  World  Trade,"  prepared  for  presentation 
at  the  American  Statistical  Association-Joint  Meetings,  6  August  1989,  Washingttm,  DC. 

The  global  nature  of  today's  business  environment  and  the  increased 
emphasis  on  quality  in  that  marketplace  has  produced  growing  interest  in 
international  quality  standards.  Such  standvds  can  play  a  major  role  in 
reducing  "non-tariff"  trade  barriers  between  nations.  Among  these  are  the 
ISO  9{)()0  series  (also  ANSI/AS()C  Q90)  which  deal  with  quality 
management  systems  and  contractual  quality  assurance  assessment. 
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Measurement  and  evaluation  methodology  standards  are  also  being 
prepared. 

Government  Accounting  Office  (GAO),  “Management  Practices,  U.S.  Comities  Improve 
Performance  Through  Quality  ^forts,"  Report  to  the  Honorable  Donald  Ritter,  House  of 
Representatives,  l^y  1991. 

Gunter,  Bert,  Contributing  Editor,  ’The  Use  and  Abuse  of  Cpk,  Part  4,"  Quality  Progress, 
May  1989. 

Gunter,  Bert,  Contributing  Editor,  "The  Use  and  Abuse  of  (Tpk,  Part  3,"  Quality  Progress, 
May  1989,  pp.  79-80. 

Gunter,  B.H.,  “The  Use  and  Abuse  of  Cpk,”  Quality  Progress,  Volume  22,  Number  1, 
1989,  pp.  72-73. 

Hall,  Thomas  D.,  Ameiock  Corporation,  "How  Close  is  s  to  o,"  Quality,  December  1991, 
p.  45. 

Hare,  Lynn  B.,  Statistical  Services,  Thomas  J.  Lipton,  Inc.,  "In  the  Soup:  A  Case  Study  to 
Identify  Contributors  to  Filling  Variability,"  Journal  of  Quality  Technology,  Volume  20, 
Numbn  1,  January  1988,  pp.  36-43. 

The  use  of  a  planned  experiment  in  a  plant  process  which  is  controlled  using 
Shewhart  control  charts  is  discussed.  The  response  is  the  filling  variation 
among  packets  of  dry  soup  mix.  It  was  desired  to  minimize  this  variation  to 
provide  more  product  uniformity  and  increase  consumer  acceptance.  The 
experimental  design  is  discuss^  as  are  summary  statistics  derived  from 
control  chart  data  and  the  analysis  and  interpretation  of  these  summary 
statistics.  The  results  are  analyzed  graphically,  without  the  use  of  formal 
statistical  tests.  They  illustrate  the  power  of  planned  experimentation  in 
identifying  opportunities  fix’  quality  and  productivity  improvement. 

Harry,  Mikel  J.,  "The  Nature  of  Six  Sigma  Quality,"  Motorola,  Inc.,  Government 
Electronics  Group. 

This  booklet  highlights  the  six  sigma  product  quality  concept  and  its 
relationships  to  Motorola's  position  in  the  marketplace. 

The  discussion  zeros  in  on  the  concept  of  six  sigma,  which  advocates  that 
there  are  strong  relationships  between  |xoduct  nonconformities  or  defects 
and  product  yield,  reliability,  cycle  time,  inventory,  schedule,  and  so  on. 

As  the  number  of  defects  found  during  manufacture  increases,  the  number 
of  sigmas  decreases.  In  other  words,  tiie  larger  the  sigma  value,  the  better 
the  ixoduct  quality-and  vice  versa.  Although  the  ultimate  aspiration  is  zero 
defects,  the  threshold  of  excellence  is  six  sigma  quality. 

Interestingly,  six  sigma  quality  is  estimated  assuming  typical  shifts  and 
drifts  in  the  average.  In  this  sense,  99.99966  percent  capability  at  the  part 
and  process  step  levels  is  an  intermediate  target  toward  ^e  ideal  of 
perfection.  This  may  be  illustrated  by  considering  a  product  that  contains 
300  parts  and  the  related  manufacturing  process  that  consists  of  say,  500 
individual  steps.  A  six  sigma  capability  at  the  part  and  process  step  levels 
would  ensure  a  final  rolled  throughput  yield  of  99.73  percent.  This  would 
be  to  say,  out  of  every  10,000  units  of  product  manufactured,  there  would 
be  9973  units  that  would  be  produced  completely  free  of  nonconformities. 

Of  coixse,  this  example  assumes  that  each  part  and  process  step  possesses 
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only  one  opportunity  for  nonconformance,  that  all  parts  and  steps  are 
independent,  and  that  nonconformities  are  randomly  distributed. 

The  notion  of  variation  is  presented  as  the  number  one  enemy  of  quality, 
yields,  and  costs.  It  must  be  arrested  and  ultimately  eliminated  in  order  to 
achieve  best  in  class.  By  attacking  variation  during  the  design  phase,  within 
suppliers'  processes,  and  within  our  own  processes,  six  sigma  product 
quality  can  be  achieved.  In  doing  so,  the  foundation  of  excellence  is  laid. 

The  discussion  also  focuses  on  a  more  statistically  based  understanding  of 
the  six  sigma  program.  It  describes  the  arithmetic  mean  (^),  standard 
deviation  o  and  practical  uses  of  the  normal  distribution.  In  particular,  the 
rationale  for  making  quality  and  ^eld  estimates  under  the  assumption  of  a 
1.5  o  shift  in  the  mean  is  emphasized.  Based  on  the  statistical  perspective, 
the  product  and  process  engineering  viewpoints  are  brought  into  focus  by 
means  of  analytical  examples.  Through  the  discussion  and  examples, 
insights  are  developed  as  to  the  objectives  of  the  six  sigma  program: 
enhanced  product  quality,  yield,  and  cost-all  of  which,  in  turn,  improve 
customer  satisfaction. 

Index  and  Directory  of  U.S.  Industry  Standards,  Volume  I,  "Subject  Index,"  and  Volume 

n,  "Society/Numeric  Listing  ANSI  Number  Concordance  Society  Directory,"  Information 

Handling  Services,  Englewood,  CO,  1985. 

Industrial  Fasteners  Institute,  pamphlets. 

"Industry  Survey  to  Measure  Quality  Management,"  Defense  Daily,  6  February  1992, 

p.  202. 

Price  Waterhouse  and  AAI  CtHporation  today  announced  the  start  of  a  joint 
survey  of  aerospace  and  defense  companies  on  the  issue  of  quality 
management 

Called  TQM/100,  the  study  seeks  to  identify  and  measure  total  quality 
management  standards  and  trends  across  a  broad  range  of  business  activities 
while  giving  participating  companies  confidential  access  to  the  survey's 
findings  to  compare  their  quality  efforts  with  those  of  first-rank  and  other 
companies  in  the  study. 

The  study  is  expected  to  encompass  30  to  50  aerospace  and  defense 
companies  with  DoD  or  NASA  sales  of  more  than  $100  million.  So  far, 
about  20  companies,  including  Grumman,  TRW,  Northrop,  and  General 
Electric,  have  agreed  to  participate.  The  cost  per  participant  is  $25,000. 

Kane,  Victor  E.  (Ford  Motor  Company),  “Process  Capability  Indices,”  Journal  of  Quality 

Technology,  Volume  18,  Number  1,  January  1986,  pp.  41-52. 

The  capability  indices  Cp,  CPU,  CPL,  k  and  Cpk  are  presented  and  related 
to  process  parameters.  These  indices  are  shown  to  form  a  complementary 
system  of  measures  of  process  performance,  and  can  be  used  with  bilateral 
and  unilateral  tolerances,  with  or  without  target  values.  A  number  of 
Japanese  Industries  currently  use  the  five  indices  and  the  U.S.  automotive 
industry  has  started  using  these  measures  in  a  number  of  areas.  Various 
applications  of  the  indices  are  discussed  along  with  statistical  sampling 
considerations. 


E-7 


Kilpatrick,  Philip  S.,  Alliant  Techsystems,  Inc.,  “Benefits  of  SPC  Application  to  Program 
Management  for  120mm  Tank  Ammunition,”  viewgraphs,  American  Defense  Prqmredness 
Association,  2nd  Annual  Statistical  Process  Control  Symposium,  Hunt  Valley,  MD, 
4  March  1992. 

Kowalick,  Jim,  “QFD,  DOE  and  Producibility,”  Renaissance  Leadership  Institute, 
Renaissance,  CA. 

Lamprecht,  James  L.,  "ISO  9000  Implementation  Strategies,”  Quality,  November  1991, 
pp.  14-17. 

The  race  is  on  here  in  the  U.S.  for  ISO  certification  in  Europe. 

Lang,  Ron,  “Achieving  Manufacturing  Excellence,”  viewgraphs.  National  Center  for 
Manufacturing  Science  (NCMS),  1993. 

Levi,  Raffaello  (Polytechnical  University  of  Turin,  Italy),  “Cautions  for  Tachuchi  Lovers,” 
Marufacturing  Engineering,  March  1993,  page  16. 

Lindley,  Dennis  V.(retired)  and  Nozer  D.  Singpurwalla  (The  George  Washington 
University),  “On  the  Evidence  Needed  to  Reach  Agreed  Action  Between  Adversaries,  With 
Application  to  Acceptance  Sampling,”  Journal  of  the  American  Statistical  Association, 
Volume  86,  Number  416,  December  1991. 

Two  decision  makers  disagree  about  a  quantity  of  interest  to  them  both. 

One  of  them,  the  “consumer,”  has  a  choice  of  two  decisitxis  that  are  affected 
by  the  quantity.  The  other,  the  “manufacturer,”  offers  to  perform  an  agreed 
type  of  experiment  that  it  is  hoped  will  change  the  consumer’s  view  of  the 
quantity  and  hence  the  decision.  This  article  is  devoted  to  the  evaluation  of 
how  much  experimentation  should  be  done.  Binomial,  Poisson,  and 
normal  likeliht^s,  together  with  their  conjugate  utilities  and  probabilities, 
are  considered  and  illustrated  hy  numerical  cases.  The  scenario  considered 
here  arises  in  applications  to  quality  control,  bidding,  drug  testing, 
marketing,  and  sales. 

Liuzza,  Carmen  J.  and  Paul  A.  Roediger,  "ABC,  Joint  AMC/ADPA  SPC  Meeting," 
viewgraphs,  ARDEC,  22  October  1991. 

Liuzza,  Carmen  J.,  Paul  A.  Roediger  (Picatinny  Arsenal),  and  Edward  G.  Schilling 
(Rochester  Institute  of  Technology),  "ABC:  A  Sampling  Plan  to  Attain  Process  C^ontrol  and 
Capability." 

Lorber,  Si  and  Bill  Kracov,  "(^ality  Program  Requirements  for  the  90's,"  May  1992. 

Lorber,  Si  and  Bill  Kracov,  DoD  Quality  Program  Requirements  for  the  90’s,  viewgraphs. 
Spring  1992. 

Lowell,  Steve,  "DoD  Non-Government  Standards  Adoption  I’rogram  Tutorial,  Past, 
Present,  and  Future,"  viewgraphs.  Office  of  the  Assistant  Secretary  of  Defense  for 
Production  and  Logistics,  Washington,  DC. 

MacDonald,  B.A.  and  M.V.  Petty,  "List  of  Quality  Standards,  Specifications  and  Related 
Documents,"  10  February  1987. 
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Marusich,  Kerry,  Alliant  Tcchsystcms,  Inc.,  "Managing  With  Control  Charts,"  Second 
Annual  American  Defense  Preparedness  Association,  Statistical  Process  Control 
Symposium,  Hunt  Valley,  MD,  1-4  March  1992. 

Management  data  varies  over  time  just  as  data  relating  to  product  quality 
characteristics  in  a  production  environment  does.  Dr.  W.  Edwards  Denting 
says  that  great  opportunities  exist  for  the  use  of  Shewhan  control  charts  in 
areas  of  supervision  and  management,  and  that  the  job  of  management  is 
one  of  prediction.  Yet,  managers  make  critical  decisions  (predictitms)  with 
intuition  or  'gut  feel'  being  the  rationale,  rather  than  by  analyzing  data  in 
time  series.  'This  meth^  of  operation  causes  lost  opportunities  for 
improvement  and/or  damage  to  the  system,  resulting  in  lower  quality  and 
increased  costs.  The  purpose  of  this  paper  is  to  demonstrate  the  importance 
using  Shewhart  control  charts  for  analyzing  management  data,  as  well  as  the 
importance  of  viewing  production  globally,  rather  than  as  a  set  of 
independent  processes. 

National  Center  for  Manufacturing  Sciences,  "Standards  Come  to  the  Fore  as  a  World- 
Class  Competitive  Force,"  Focus,  March  1992. 

National  Institute  for  Standards  and  Technology  (NIST),  "Standards  Developing 
Qrgaitizations  (Abbreviations  and  Directory),"  January  1991. 

Navai  Surface  Warfare  Center,  Crane  Division,  Procurement  Quality  Handbook,  June 
1993 

Ott,  Ellis  R.  (The  State  University  of  New  Jersey,  Rutgers),  and  Edward  G.  Schilling 
(Rochester  Institute  of  Technology),  "On  Sampling  to  Provide  a  Feedback  of  Information," 
Chapter  6,  Process  Quality  Control,  McGraw-Hill  Publishing  Co.,  Second  Edition, 
pp.  123-129. 

Ptq),  Geza  M.,  "Measuring  to  AQL  Ineffective,"  letter.  Quality  Progress,  April  1992. 

Pap,  Geza  M.,  "AQL  Elimination,  EDA  Task  and  Related  Activities,"  viewgraphs, 
Picatiiuiy  Arsenal,  NJ,  17  March  1992. 

Pap,  Geza  M.,  AMSMC-QAH  ^),  Picatinny  Arsenal,  NJ,  "Elimination  of  Acceptable 
(^{^ty  Levels  (AQLs)  from  Military  Specification,"  viewgraphs,  presented  to  AMCCOM, 
28  January  1992. 

Pecht,  Michael  (University  of  Maryland)  and  Edward  Hakim  (Fort  Monmouth,  NJ),  "The 
Future  of  Military  Standards,  A  Focus  on  Electrcmics." 

With  cut-backs  in  the  militapr  and  the  changing  international  economic 
environment,  there  is  a  growing  perception  that  the  military  must  rely  on 
commercial  industries  in  order  to  afford  the  next  generation  of  high 
performance  and  high  reliability  military  systems.  What  is  not  clear  is 
whether  an  industrial  base  for  the  defense  can  be  provided  and  maintained 
without  dramatic  chang  ;s  in  the  operation  of  both  the  government  (DoD) 
and  commercial  industry.  This  paper  examines  one  issue  associated  with 
this  question — ^whether  commercial  standards  can  effectively  supplant 
milit^  standards  and  still  provide  high  performance  and  reliability  for 
military  missions. 
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Pennucel,  Nicholas  J.  (Calcomp,  Anaheim,  CA),  “What  are  Cn  and  Cpk?,”  Quality,  August 
1986. 

Once  a  process  is  in  statistical  control,  it  doesn’t  mean  the  work  is  done.  It 
is  only  the  first  step.  The  next  question  is,  “Will  the  product  meet  the  sp^, 
and  how  well  will  it  meet  the  spec?,*’  Cp  and  Cpk,  respectively,  can  provide 
the  answers. 

Placek,  Chester,  "Fasteners  Group  Issues  Guide  to  Recommended  SPC  Practices  for 
Fastener  Industry,"  Quality,  April  1991,  p.  13. 

Statistical  Process  Control  (SPC)  guide  published  by  industrial  Fasteners 
Institute  should  help  manufacturers  of  fasteners  and  formed  parts  achieve  a 
common  basis  for  assuring  product  quality. 

Process  Action  Team,  Drefi  Charter  on  Military  Specifications  and  Standards,  AMCRD-IC 
c50N  Engineering,  August  1993. 

Process  Action  Team,  Elimination  of  Fixed  Defect  Levels  (Acceptable  Quality  Levels/Lot 
Tolerance  Percent  Defective)  from  Military  Specifications,  Final  Report,  MCRDAC  PS, 
24  September  19%. 

This  report  summarizes  the  efforts  of  a  joint-service/agency  Process  Action 
Team  (PAT)  formed  to  determine  the  most  effective  approach  to  remove 
Acceptable  (^aUty  Levels  (AQLs)  and  Lot  Tolerance  Percent  Defective 
(LTPDs)  from  military  specifications  as  acceptance  criteria.  In  additim,  the 
PAT  leoxnmended: 

Standard  methods  to  replace  such  statements. 

Uniform  methods  to  convey  expectations  of  total  compliance  to 
specification  requirements,  and 

Methods  to  measure  quality  of  product  without  specifying  risk 
statements. 

The  PAT  effort  embodied  the  basic  concepts  of  Total  Quality  Management 
(T(^M)  as  implemented  within  OSD.  llie  PAT  initiated  its  task  with  a 
review  of  previous  efforts  which  had  addressed  the  same  subject;  in 
particular,  the  final  report  of  the  Joint  Services  Working  Group  on 
Elimination  of  Fixed  Defect  Levels  dated  13  July  1988.  Subsequently,  the 
PAT  conducted  an  in-depth  analysis  of  die  history,  use  and  misuse  of  AQLs 
and  LTPDs,  so  that  appropriate  benefits  of  their  use  would  not  be  lost  while 
the  negative  aspects  were  being  removed. 

The  PAT  recommends  a  fundamental  shift  of  focus  of  verification  and 
acceptance  activities  fiom  end-of-line  inspections  and  tests  to  in-line  or  off¬ 
line  manufacturing  process  controls,  llie  desired  outcome  is  to  convey 
clearly  an  expectation  of  100  percent  compliance  to  well  defined,  specified 
requirements  while  preserving  the  right  of  Government  to  use  appropriate 
ve^cation,  inclutfing  in-plant  samplmg  to  establish  confidence  in  meeting 
requited  quality.  Spe^c  recommendations  include: 

Remove  AQL/LTPD  terminology  and  similar  expressions  from 
military  specifications.  The  use  of  sampling  plans  and  statistical 
techniques  is  not  prohibited  by  the  removal  of  such  terminology. 

Immediately  revise  MIL-STDs  961,  962,  and  490  to  permit 
accomplishrnent  of  the  first  recommendation. 
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In  a  phased  orderly  manner,  r^uire  that  new  or  newly  revised  product 
q)ecifications  meet  these  requirements  priOT  to  publication. 

Sponsor  further  action  to  develop  acquisition  language  and  techniques 
such  that  DoD  can  successfully  inclement  the  quality  policy  with  long 
term  focus  on  process  control  and  continuous  improvement. 

"The  Quality  Glossary,"  Quality  Progress^  February  1992,  pp.  20-29. 

Rade,  Leonard  G.  (Peripheral  Conqmnents  International),  “Enhance  Product  Development 
by  Using  Capability  Indexes,”  Quality  Progress^  April  1989,  pp.  38-41. 

Reiss,  Fred  J.,  GEC-Maiconi  Electronic  Systems  Corporadon,  "Graphical  Analysis  Using 
ProbabiUty  Paper,"  2nd  Annual  Technical  Workshop  on  Quality  Control  and  Statistics, 
Princeton  University,  Princeton,  NJ,  4  December  1971. 

Rogers,  Lee,  P.E.,  The  Rationale  and  Plan  for  Eliminating  Fixed  Allowable  Levels  of 
Defects  from  Military  Specifications  (MIL-specs)  and  DoD  Procurements,  6  June  1991, 
Di^t 

Riddell,  Frederick  R.,  "Notes — Meeting  With  Ed  Schilling,"  Rochester  Institute  of 
Technology,  Rochester,  NY,  28-29  October  1991. 

Rydeski,  James  A.,  Alliant  Techsystems,  "Management  and  SPC,"  viewgraphs,  presented 
at  American  Defense  Prepar&iness  Association,  Second  Annual  Statistical  Process  Control 
Symposium,  Hunt  Valley,  MD,  3  March  1992. 

Rydeski,  James  A.,  Alliant  Techsystems,  “A  SPC  Rating  System,  An  Effective 
Communication  Tool,”  presented  at  The  Third  Annual  SPC  Division  of  ADPA’s  SPC 
Symposium  “SPC:  A  Management  Strategy,  17  March  1993. 

Schilling,  Edward  G.  (Rochester  Institute  of  Technology,  Rochester,  NY),  “The  Transition 
from  Sampling  to  SPC,”  Quality  and  Statistics:  Total  Quality  Management,  ASTM  STP 
1209,  Milton  J.  Kowalsewski,  Jr.,  Ed.,  American  Society  for  Testing  and  Materials, 
Philadelphia,  1994. 

Modem  quality  control  practice  emphasizes  the  importance  of  process 
control  in  the  creation  of  quality  product.  When  the  process  is  in  control,  it 
is  possible  to  characterize  the  jncduct  fcxrthcoming  from  the  process,  for  Ae 
particular  characteristic  controlled,  at  the  point  in  the  process  at  which  the 
control  is  instituted.  When  the  process  is  out  of  control,  the  relationship 
between  product  and  process  is  lost.  It  is  then  necessary  to  go  rely  on 
acceptance  sampling  procedures  to  characterize  the  process.  By  combining 
the  power  of  process  control  with  the  assurance  of  acceptance  sampling,  the 
process  experimentation  necessary  for  continual  improvement  can  be 
undertaken  with  minimal  risk  to  Ae  consumer.  An  approach  has  been 
developed  to  transition  from  a  process  lacking  control  an^or  capability  to  a 
controlled  process  with  a  C^k  value  1.33  indicating  that  the  process  is 
operating  at  an  average  level  four  signna  ih»n  the  specification(s). 

Schilling,  Edward  G.,  Letter,  ABC  for  IQUE,  1  February  1992. 

Schilling,  Edward  G.,  Scientific  Services  Program,  "An  ABC  Sampling  Plan  to  Attain 
Process  Control  and  Capability,"  Armament  Research,  Development  and  Engineering 
Center,  Picatinny  Arsenal,  NJ,  28  February  1991. 

An  ABC  Sampling  Plan  for  acceptance  of  products  and  processes  is 
presented  which  incorporates  three  successive  stages  to  move  from  an 
uncontrolled  process  to  a  process  under  statistical  control  with  Cpk  =  1.33. 
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Protection  is  afforded  in  terms  of  the  AQQL  of  sampling  plans  run  in 
parallel  with  implementation  of  statistical  process  control.  Switching  from 
state  to  stage  is  in  terms  of  stringent  requirements  in  terms  of  capability  and 
control.  The  plan  is  structured  to  be  of  maximum  educational  benefit  to  the 
user  in  prcxnc^g  continual  improvement. 

Schilling,  E.G.,  Rochester  Institute  of  Technology,  "New  ANSI  Versions  of 
Mn^-STD-414  and  MfL-STD-lGSD,"  Naval  Research  Logistics  Quarterly,  Volume  32, 
1985,  pp.5-9. 

When  acceptance  san:q}ling  plans  are  applied  to  measurement  characteristics, 
a  choice  between  application  of  a  variables  or  an  attributes  plan  must  be 
made.  With  the  recent  revision  by  the  American  Society  of  Quality  Control 
for  the  American  National  Standards  Institute  of  the  ANSI/ASQC  Z1.9 
variables  system  and  the  ANSI/ASQC  Z1.4  attributes  system,  the  standards 
have  now  been  matched  so  that  it  is  possible  to  move  between  them.  This 
article  discusses  exploitation  of  the  resulting  synergistic  relationship  to 
achieve  trmre  rational  and  more  effective  acceptance  sampling. 

Schilling,  Edward  G.,  General  Electric  Company,  "Revised  Attributes  Acceptance 
Sampling  Standard — ANSI/ASQC  Z1.4  Journal  of  Quality  Technology,  Volume 

14,  Number  4,  October  1982,  pp.  215-219. 

The  United  States  National  Standard  ANSI/ASQC  Z1.4  (1981)  is  a  revised 
version  of  the  nulitary  attributes  sampling  system  MIL-STD-105D.  This 
standard  has  undergone  revisions  to  modernize  terminology  and  to 
emphasize  the  system  aspect  of  the  procedure.  In  addition,  it  has  b^n  made 
even  more  compatible  with  its  variables  counterpart  ANSI/AS(^  Z1.9 
(1980),  which  was  derived  from  MIL-STD-414.  The  development  and 
nature  of  these  revisions  are  described. 

Schilling,  Edward  G.,  General  Electric  Company,  'Two  New  ASCJC  Acceptance  Sampling 
Standards,"  Quality  Progress,  March  1983,  pp.  14-17. 

Two  recently  revised  standards  now  fit  together  to  allow  more  effective 
sampling. 

Schilling,  Edward  G.,  Rochester  Institute  of  Technology,  "Product  Oriented  Quality 
Control  and  Assurance,"  invited  paper  from  Proceedings  of  the  35th  EOQ  Annual 
Corference,  17-21  June  1991,  Prague,  Czechoslovakia,  pp.  376-382. 

This  paper  describes  the  advantages  and  desirability  of  the  use  of  statistical 
process  control  techniques  in  characterizing  the  product,  while  pointing  out 
the  areas  in  which  such  techniques  are  inappropriate.  Strategies  are 
proposed  which  incorporate  both  process  control  and  acceptance  sampling 
to  achieve  the  efficiencies  offered  by  proper  application  of  toth  procedures. 

Schilling,  Edward  G.,  Paul  A.  Miller  (Rochester  Institute  of  Technology),  and  Dan  J. 
Sommers  (General  Electric  Company),  "Section  25,  Acceptance  Sampling,"  reprinted  by 
permission  from  Quality  Control  Handbook,  edited  by  J.  M.  Juran  and  Fr^  M.  Gryna, 
Jr.,  McGraw-Hill  Book  Company,  1988. 

Schilling,  E.G.  (Rochester  Institute  of  Technology)  and  H.F.  Dodge  (Rutgers  University), 
"Procedures  and  Tables  for  Evaluating  Dependent  Mixed  Acceptance  Sampling  Plans," 
Technometrics,  Volume  1 1,  Number  2,  May  1969. 

This  paper  gives  procedures  and  tables  for  evaluating  the  operating 
characteristic  curves  and  associated  measures  of  dependent  mixed 
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acceptance  sampling  plans  for  the  case  of  single  specification  limit  and 
known  standard  deviation,  assuming  a  normal  distribution.  Joint 
probabilities  necessary  for  evaluating  these  measures  are  derived  and 
methods  to  facilitate  their  computation  are  provided.  A  useful  generalized 
dependent  plan  is  also  present^,  using  two  attributes  acceptance  numbers 
rather  than  one.  Tables  of  joint  probabilities  necessary  for  evaluation  of 
mixed  plans  are  presented  for  first  sample  sizes  of  4,  S,  8,  and  10, 
acceptance  numbers  of  0, 1,  and  2  and  various  percentages  defective. 

Schilling,  Edward  G.,  Rochester  Institute  of  Technology,  "Acceptance  Sampling  in  Quality 
Control,"  ASQC  Quality  Press,  Marcel  Dekker,  Inc.,  New  York  and  Basel,  Milwaukee, 
pp.  76-107. 

Schilling,  Edward  G.,  "The  Role  of  Statistics  in  the  Management  of  Quality,"  based  on  the 
Shewhan  Medal  acceptance  speech  made  at  the  Honors  and  Awards  Breal^ast  at  the  38th 
Annual  Qualitv  Congress,  Quality  Progress,  August  1984,  pp.  32-35. 

Schweiker,  Jane,  Independent  Consultant,  "ASTM  and  the  Department  of  Defense," 
ASTM  Standardization  News,  September  1991. 

Shainin,  Dorian  and  Peter  D.  Shainin,  "Statistical  Process  Control,"  Juran's  Quality 
Control  Handbook,  Section  24, 4th  Edition,  McGraw  Hill,  1988. 

Singpurwalla,  Nozer  D.,  The  George  Washington  University,  “Design  by  Decision 
Thecny:  A  Unifying  Perspective  on  Taguchi’s  Approach  to  Quality  Engineering,”  to  appear 
in  Reliability  and  Decision  Making,  Elsevier  Science,  pp.  267-272. 

We  argue  that  an  encompassing  perspective  on  Taguchi’s  approach  to 
quality  engineering  is  provided  by  statistical  decision  theory.  This  theory 
deals  with  decision  mal^g  in  the  face  of  uncertainfy,  with  or  without  partial 
information,  and  prescribes  that  an  optimum  decision  is  one  that  maximizes 
(minimizes)  expected  utility  (loss).  The  role  of  experimental  design  is  to 
enable  one  to  obtain  partial  information  about  unsown  quantities  in  an 
efficient  manner.  When  viewed  as  such,  much  of  what  Taguchi  advocates, 
including  his  proposals  for  tolerance  design,  gets  streamlin^  and  integrated 
as  a  comprehensive  package.  An  advantage  of  the  proposed  perspective  is  a 
better  delineation,  and  possible  elimination  of  some  areas  of  controversy. 
Furthermore,  it  helps  us  focus  on  issues  additional  to  those  pertaining  to 
experimental  design  and  thereby  provides  opportunities  for  new  research  to 
expand  and  to  build  upon  Taguchi’s  sensible  plan. 

Singpurwalla,  Nozer  D.,  The  George  Washington  University,  “A  Bayesian  Perspective  on 
Taguchi’s  Approach  to  Quality  Engineering  and  Tolerance  Design,”  IIE  Transactions, 
Volume  24,  Number  5,  November  1992,  pp.  18-27. 

An  impetus  for  the  new  revolution  in  qualify  technology  has  been  Professor 
Genichi  Taguchi’s  approach  to  quality  engineering,  best  exemplified  by  his 
call  for  off-line  quality  control.  However,  much  of  the  literature  on  this 
topic  appears  to  be  fragmented  between  engineering,  statistics  and  quality 
control  journals,  each  emphasizing  a  point  of  view  that  is  pertinent  to  its 
readership.  A  consequence  of  the  above  is  that  there  has  been  some 
difficulty  in  developing  an  appreciation  for  the  totality  of  the  approach,  its 
key  ingi^ients,  and  the  several  excellent  contributions  of  many  others  in 
tins  important  subject.  In  this  paper,  we  attempt  to  help  alleviate  this 
difficulty  by  pointing  out  that  an  encompassing  perspective  on  Taguchi’s 
philosophy  can  be  provided  by  statistical  decision  analysis.  The  subject 
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deals  with  decision  making  in  the  face  of  partial  or  no  information,  and 
prescribes  that  an  optimum  decision  is  one  ^at  maximizes  expected  utility. 

The  role  of  experimental  design  is  to  obtain  partial  information  about  the 
unknown  quantities  in  an  efficient  manner.  When  viewed  as  such,  much  of 
what  Taguchi  prescribes,  including  his  proposals  for  tolerance  design,  gets 
stieamlii^  and  integrated  as  a  coti^irehensive  package. 

Society  of  Automotive  Engineers  (SAE),  Report  to  Aerospace  Industry,  1991  SAE 

Cooperative  Engineering  Program,  January  1991. 

Standardization  Management  Activity  Seminar,  information,  17  March  1994. 

Stratton,  Brad,  ed.,  "Quality  Goes  to  War:  An  Overview,"  Quality  Progress,  December 

1991,  pp.  18-42. 

Stratton,  John  H.,  "What  is  the  Registrar  Accreditation  Board?,"  Quality  Progress,  January 

1992,  pp.  67-69. 

Third  Party  Accreditation'  Acceptance,  Applications,  and  Implications,  Bulletin  from  The 

Performance  Review  Institute's  First  Conference,  Holiday  Inn,  Washington,  DC, 

15  October  1991. 
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Assurance  and  Testing,  AMC),  Executive  Director  (Reliability,  Maintainability  and  (^ality 
Assurance,  OASN  (S&L)RM&QA),  Director  ((Contracting  and  Manufacturing,  AF^DC), 
and  Executive  Direaor  ((^ality  Assurance,  DLA-(^,  1 1  March  1987. 

Mittino,  John  A.,  Deputy  Assistant  Secretary,  Acquisition  and  Logistics  (PS/IPQ), 
“Eliminating  Acceptable  Quality  Levels  (AQL)  and  Lot  Tolerance  Percent  Defective  (LTPD) 
from  Military  and  Feder^  Specifications,"  Memorandum  for  Assistant  Secretary  of  the 
Army  (Research,  Development  and  Acquisition),  Assistant  Secretary  of  the  Navy 
(Shipbuilding  and  Logistics),  Assistant  Secretary  of  the  Air  Force  (Acquisition),  and 
Director  (Defense  Logistics  Agency),  16  June  1987. 

Pucilowski,  Joseph  J.,  Jr.,  Acting  Deputy  Chief  of  Staff  for  Concurrent  Engineering, 
Department  of  the  Army,  Headquarters,  U.S.  Army  Materiel  Command,  AMCCE-QE, 
"AQL/LTPD  Policy,"  Memorandum  for  Commander,  U.S.  Army  Laboratory  Command, 
Materials  Technology  Laboratwy,  Alexandria,  VA,  17  May  1991. 
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Torelli,  Nicholas  M.,  Jr.,  Deputy  Assistant  Secretary  (Production  Resources),  “Review  of 
DoD  (Quality  Program,”  Memorandum  for  OASA  (RDA),  Mr.  Burdt;  OASN  (RD&A)  PR, 
Mr.  Willoughby;  SAIVAQX,  Maj.  Gen.  Condon;  and  DLA-Q,  Brig.  Gen.  Hruskocy, 
IS  January  1993. 

Torelli,  Nicholas  M.,  Jr.,  Deputy  Assistant  Secretary  (Production  Resources),  “Application 
of  DoD  Quality  Military  Standards,”  Memorandum  for  the  Under  Secretary  of  Defense 
(Acquisition),  22  December  1992. 

Torelli,  Nicholas  M.,  Jr.,  Deputy  Assistant  Secretary  (Production  Resources),  “Use  of 
International  Standards  (>ganization  (ISO)  9(X)0  Series  Quality  Assurance  Standards,” 
Memorandum  for  Assistant  Secretary  of  the  Army  for  Research  Development,  and 
Acquisition;  Assistant  Secretary  of  the  Navy  for  Research,  Development,  and  Acquisition; 
and  Commander,  Defense  Contract  Management  Command,  16  September  1992. 

Twelli,  Nicholas  M.,  Jr.,  Deputy  Assistant  Secretary  (Production  Resources),  “Use  of 
Statistical  Process  Control  (SPC)  Techniques  vs.  Acceptable  Quality  Levels  (AQLs),” 
Memorandum  for  Executive  Director,  Quality  Assurance,  Defense  Logistics  Agency, 
25  August  1^2. 

Letter  to  be  Attached  to  all  Picatinny  Solicitadons/C^ntracts  Which  Have  AQL  Free  TDPs 
(0-1  Plans),  Fourth  Draft,  U.S.  Army  Armament  Munitions  and  Chemical  Command, 
SDecembCT  1991. 

Todaro,  John  B.,  Industrial  Productivity  and  Quality,  PR/I^,  "Fixed  Allowable  Level  of 
Defects  Handbook,"  Memorandum  for  Director,  Standardization  and  Data  Management, 
15  October  1990. 

Wade,  James  P.,  Jr.,  Assistant  Secretary  of  Defense,  Acquisition  and  Logistics  (PS/IP(^, 
“Achieving  Continuous  Quality  Improvement,”  Memorandum  for  Assistant  Secretary  of  the 
Army  (Research,  Development  and  Acquisition),  Assistant  Secretary  of  the  Navy 
(Shipbuilding  and  Lopstics),  Assistant  Secretary  of  the  Air  Force  (Research,  Development 
and  Logistics),  and  Directors  of  the  Defense  Agencies,  16  October  1986. 

Westcott,  Edmund  J.,  Assistant  Deputy  Chief  of  Staff  for  Research,  Development  and 
Engineering — ^Technology  and  Engineering,  “Standardized  Contract  Requirement 
Specification  for  Statistical  Process  Control  (SPC),”  Memorandum  for  Distribution, 
Headquarters,  U.S.  Army  Materiel  Command,  Alexandria,  VA,  20  May  1993. 

Wyatt,  John  A.,  Director,  Defense  Quality  and  Standardization  Office,  “Elimination  of 
Fixed  Defect  Levels  from  all  Military  and  Federal  SpeciEcations  Prepared  by  DoD 
Activities,  Office  of  the  Secretary  of  Defense,  Production  and  Logistics,  Memorandum  for 
Departmental  and  DoD  Agency  Standardization  Offices,  28  March  1989. 

Yurcisin,  Peter,  Director,  Standardization  and  Data  Management,  OASD,  P&L,  "Fixed 
Allowable  Level  of  Defects  Handbook,"  Memorandum  for  Acting  Director,  Industrial 
Productivity  and  (Quality,  5  October  1990. 

Yurcisin,  Peter,  Acting  Deputy  Assistant  Secretary  of  Defense  (TQM),  "Use  of  Acceptable 
Quality  Levels  (AQLs)  and  Lot  Tolerance  Percent  Defectives  (LlTDs)  in  Standardization 
Documents,"  Producuon  and  Logistics,  Memorandum  for  Standardization  Executives  of 
the  Military  Departments  and  Defense  Logistics  Agency,  16  August  1989. 
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STANDARDS 


ANSl/ASQC 

American  National  Standard  for  Plastic  Bathtub  Units,  ANSI  Z124.1a-1990,  Addendum  to 
ANSI  Z124. 1-1987, 6  March  1990. 

American  Society  for  Quality  Control,  ANSI  Z1.1-J985  Guide  for  Quality  Control  Charts, 
ANSI  ZI 2-1985  Control  Chart  Method  of  Analyzing  Data,  ANSI  Z1 3-1985  Control 
Chart  Method  of  Controlling  Quality  During  Production,  Milwaukee,  WI. 

American  Society  for  Quality  Control,  ASQC  Standard  Project  43,  American  National 
Standard,  Introduction  to  Attribute  Sampling,  Milwaukee,  WI. 

American  Society  for  Quality  Control,  D^nitions,  Symbols,  Formulas,  and  Tables  for 
Control  Charts,  ANSI/ASQC  Al-1987,  American  National  Standard,  November  1987. 

American  Society  for  Quality  Control,  Terms,  Symbols,  and  Definitions  for  Acceptance 
Sampling,  ANSI/ASQC  A2-1987,  American  National  Standard,  November  1987. 

American  Society  for  Quality  Control,  Quality  Systems  Terminology,  ANSI/ASQC 
A3- 1987,  November  1987. 

Quality  Management  and  Quality  Assurance  Standards— Guidelines  for  Selection  and  Use, 
ANSI/ASQC  Q90-1987,  American  Society  for  Quality  Control,  Milwaukee,  WS,  approved 
8  June  1987,  American  National  Standards  Institute,  Inc. 

Quality  Management  and  Quality  Assurance  Standards— Guidelines  for 
Selection  and  Use  provides  guidelines  for  the  selection  and  use  of  Standards 
Q91,Q91,Q91,andQ94, 

Quality  Systems—Model  for  Quality  Assurance  in  DesignIDevelopment,  Production, 
Installation,  and  Servicing,  ANSI/ASQC  Q91-1987,  American  Society  for  Quality  Control, 
Milwaukee,  WS,  approv^  19  June  1987,  American  National  Standard  Institute,  Inc. 

Quality  Systems—Model  for  Quality  Assurance  in  DesignIDevelopment, 
Production,  Installation,  and  Servicing  specifies  quality  system 
requirements  for  use  where  a  contract  between  two  parties  requires  the 
demonstration  of  a  supplier’s  capability  to  design  and  supply  product. 

Quality  Systems—Model  for  Quality  Assurance  in  Production  and  Installation, 
ANSI/AS(^  Q92-1987,  American  Society  for  Quality  Control,  Milwaukee,  WS,  approved 
15  June  1987,  American  National  Standards  Instiuite,  Inc. 

Quality  Systems—Model  for  Quality  Assurance  in  Production  and 
Installation  specifies  quality  system  requirements  for  use  where  a  contract 
between  two  parties  requires  the  demonstration  of  a  supplier's  capability  to 
ctmtrol  the  processes  that  determine  the  acceptability  of  a  product  supplit^. 

Quality  Systems— Model  for  Quality  Assurance  in  Final  Inspection  and  Test,  ANSI/ASQC 
Q93-1987,  American  Society  for  Quality  Control,  Milwaukee,  WS,  approved  15  June 
1987,  American  National  Standards  Institute,  bic. 

Quality  Systems— Model  for  Quality  Assurance  in  Final  Inspection  and  Test 
species  quality  system  requirements  for  use  where  a  contract  between  two 
parties  requires  the  demonstration  of  a  supplier's  capability  to  design  and 
control  the  disposition  of  any  product  nonconformity  during  final  inspection 
and  test. 
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Quality  Management  and  Quality  Systems  Elements— Guidelines,  ANSI/ASQC  Q94- 1987, 
American  Society  for  Quality  Control,  Milwaukee,  WS,  approved  IS  June  1987,  American 
National  Standards  Institute,  Inc. 

Guidelines  describes  a  basic  set  of  elements  by  which  a  Quality 
Management  System  can  be  developed  and  implemented  internally. 

ANSI/EIA 

Statistical  Process  Control  Systems,  ANSI/EIA-557-1989,  Electronic  Industries 
Association,  Washington,  DC,  approv^  26  April  1989,  American  National  Standards 
Institute,  Inc.,  May  1989. 

Zero  Acceptance  Number  Sampling  Procedures  and  Tables  for  Inspection  by  Attributes  of  a 
Continuous  Manufacturing  Process,  ANSI/EIA-S84-1991,  Electronics  Industries 
Association,  Engineering  Department,  Washington,  DC,  14  June  1991. 

Zero  Acceptance  Number  Sampling  Procedures  and  Tables  for  Inspection  by  Attributes  of 
Isolated  Lots,  ANSIyEIA-585-1991,  Electronics  Industries  Association,  Engineering 
Department,  Washington,  DC,  14  June  1991. 

ANSI/AWS 

Spec^cation  for  Aluminum  Brazing,  American  Welding  Society,  ANSI/AWS  C3.7-91,  An 
American  National  Standard. 

ANSI/IPC 

General  Requirements  for  Implementation  of  Statistical  Process  Control, 
ANSI/IPC-PC'90,  Institute  for  Interconnecting  and  Packaging  Electronic  Circuits,  October 
1990. 


ASME 

Quality  Assurance  Program  Requirements  for  Fastener  Manifacturers  and  Distributors,  the 
American  Society  of  Mechanic^  Engineers  (ASME),  FAP-1-1990,  New  York,  NY,  1  May 
1990. 

ASTM 

Form  and  Style  for  ASTM  Standards,  Eighth  Edition,  13-000001-89,  Philadelphia,  PA, 
September  1989. 

Precision  and  Bias  for  Various  Applications,  Third  Edition,  03-Sl  1088-34,  Philadelphia, 
PA,  1988. 

Keyword  Listing  of  ASTM  Standards  With  Acceptance  Sampling. 

Boeing 

Advanced  Quality  System,  D 1-9000,  for  Boeing  Suppliers,  1991. 
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Defense 


Buying  Nondevelopmental  Items  (NDI),  SD-2,  Depanment  of  Defense,  Office  of  the 
Assistant  Secretary  of  Defense  for  Production  and  Logistics,  Washington,  DC,  October 
1990. 

Defense  Federal  Acquisition  Regulations  (FAR),  pp.  24, 799-24,814,  January  1992. 

Defense  Standardization  and  Specification  Program  Policies,  Procedures  and  Instructions, 
DoD  4120.3-M,  August  1978,  supersedes  DoD  4120.3-M,  dated  January  1972,  and 
Appendix  C,  Non-Government  Standards,  DoD  4120.3-M,  Draft,  November  1991. 

DoD  Interaction  with  Nongovernment  Standards  Bodies,  Defense  Standardization  and 
Specification  Program,  SD-9,  Office  of  the  Under  Secretary  of  Defense  for  Research  and 
Engineering,  Washington,  DC,  April  1984. 

In-Plant  Quality  Evaluation  (IQUE),  Defense  Logistics  Agency,  DLAM  8200.5,  Cameron 
Station,  Alexani^a,  VA,  October  19%. 

Standardization  Directory  (FSC  Class  and  Area  Assignments),  SD-1,  Defense 
Standardization  and  Specification  Program,  Revised  as  of  1  September  1991. 


Federal 

Federal  Acquisition  Regulation  (FAR),  pp.  18-901-18-927, 1  February  1992. 

Ford 

Worldwide  Quality  System  Standard  Q-IOI  For  Manufacturing  Operations  and  Outside 
Suppliers  of  Production  and  Service  Products,  Corporate  Quality  Office,  Ford  Motor 
Company,  Plymouth,  MI,  1990,  notebook  with  following: 

(^ality  System  Survey  and  Scaring  Guidelines,  15  April  1990. 

Worldwide  Supplier  Quality  Rating  System,  15  April  1990. 

The  Initital  Sample  Review  Process  for  Suppliers  to  Ford  Motor  Company, 

15  April  1990. 

Planning  for  Quality,  15  April  1990. 

Q1  Preferred  Quality  Award  for  Suppliers  to  Ford  Motor  Company, 

15  April  1990. 

Worldwide  Quality  System  Standard  Q-IOIW,  Warehouse  and  Distribution,  Ford 
Corporate  Quality  (>ffice.  Dearborn,  MI,  1991. 

Facilities  and  Tools,  Quality  System  Standard,  Ck>rporate  (^ality  Office,  May  1991. 


Ford,  Chrysler,  and  GM 

Fundamental  Statistical  Process  Control,  Reference  Manual,  Troy,  MI,  1991. 
Measurement  Systems  Analysis,  Reference  Manual,  Troy,  MI,  1990. 

(^ality  System  Standard,  Draft,  29  September  1993. 
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Industrial  Fasteners  Institute 

Recommended  Practices  for  Statistical  Process  Control.  Practical  Approach  to  Statistical 
Methods  for  the  Fastener  Industry.  Industrial  Fastener  Institute.  Cleveland.  OH.  1991. 


International  Standards  Organization  (ISO) 

Quality  Management  and  Quality  Assurance  Standards— Guidelines  for  Selection  and  Use, 
ISO  90(X).  1987. 

Quality  Management  and  Quality  Assurance  Standards— Guidelines  for  Selection  and  Use, 
Proposed  Revision,  ISO/DIS  9000-1,  ISO/TC  176/SC  2. 

Quality  Systems-^odel  for  Quality  Assurance  in  Design,  Development,  Production, 
Installation  and  Servicing,  ISO  9001, 1987. 

Quality  Systems— Model  for  Quality  Assurance  in  Design/Development,  Production. 
Installation  and  Servicing,  Proposed  Revision,  ISO/DIS  9001. 

Quality  Systems— Model  for  Quality  Assurance  in  Production  and  Installation,  ISO  9002, 
1987. 

Quality  Systems— Model  for  Quality  Assurance  in  Production,  Installation  and  Servicing, 
Proposed  Revision,  IS04)IS  9002. 

Quality  Systems— Model  for  Quality  Assurance  in  Final  Inspection  and  Test,  ISO  9003, 
1987. 

Quality  Systems— Model  for  Quality  Assurance  in  Final  Inspection  and  Test,  Proposed 
Revision.  ISO/DIS  9003. 

Quality  Management  and  Quality  System  Elements— Guidelines,  ISO  9(X)4, 1987. 

Quality  Management  and  Quality  System  Elements— Part  I:  Guidelines,  Proposed 
Revision,  ISO/DIS  9004-1 . 


JEDEC 

General  Standard  for  Statistical  Process  Control  (SPC),  JEDEC  Standard  No.  19,  July 
1988,  Electronic  Industries  Association,  Engineering  Department. 

Military  Standards,  Specifications  and  DIDs 

Corrective  Action  and  Disposition  System  for  Nonconforming  Material,  MIL-STD-1520C, 
27  June  1986,  Superseding  MIL-S'ID-1520B,  3  July  1980. 

Inspection  System  Requirements,  MIL-I-45208A,  16  December  1963,  superseding 
MIL-I-45208  (Army),  12  October  1961,  NPD  (NAVEXOS  P-1034),  Appendix  A  (In 
Part),  26  Februaiy  1960. 

Preparation  of  Military  Specifications  and  Associated  Documents,  MIL-STD-961C, 
20  May  1988,  superseding  MIL-STD-961B,  31  May  1985. 

Preparation  of  Military  Standards,  Handbooks,  and  Bulletins,  MIL-STD-962B,  20  May 
1988,  superseding  MIL-STD-962A,  26  October  1984  (2  copies). 
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Quality  Assurance  Terms  and  Definitions,  MIL-STD-109B,  4  April  1969,  superseding 
MIL-STD-109A.  30  October  1961. 

Quality  Program  Requirements,  MIL-Q-98S8A,  16  December  1963,  superseding 
MIL-Q-9858, 9  April  1959. 

Quality  System  Plan,  Picatinny  Arsenal,  SMCAR-QAH,  DIQClC-81379, 7  January  1994. 

Reliability  Testing  for  Engineering  Development,  Qualification,  and  Production, 
MIL-STD-781D,  17  October  1986,  Superseding  MIL-STD-781C,  21  October  1977, 
MIL-STT)-1635  (EC),  3  February  1978,  and  MIL-STD-2068  (AS),  21  March  1977. 

Sampling  Procedures  and  Tables  for  Inspection  by  Variables  for  Percent  Defective, 
MIL-STD-414,  11  June  1957,  superseding  ORD-M608-10,  June  1954,  NAVORD  and 
OSTD  80, 8  May  1952. 

Sampling  Procedures  and  Tables  for  Inspection  by  Attributes,  MIL-STD-105E,  10  May 
1989,  superseding  MIL-STD-105D,  29  April  1963. 

Single-  and  Multi-Level  Continuous  Sampling  Procedures  and  Tables  for  Inspection  by 
Attributes,  Functional  Curves  of  the  Continuous  Sampling  Plans,  MIL-STD-1235C, 
15  March  1988,  superseding  MIL-STD-1235A-1,  28  June  1974,  MIL-STD-1235B, 
10  December  1981. 

Specification  Practices,  MIL-STD-490A,  4  June  1985,  superseding  MIL-STD-490, 
30  October  1968. 

Systems  Engineering,  Draft  Military  Standard,  MIL-STD-499B,  superseding 
MIL-STD-499A,  1  May  1974. 


Military/DoD  Handbooks 

Contractor  Performance  Certification  Program,  Design  Contractors,  Revision  A,  U.S. 
Army  Armament  Munition  and  Chemical  Ctmamand,  April  1991. 

Exerrqrlary  Facilities  (EF),  Draft  EF  Regulation  as  of  1  October  1991,  Department  of 
Defense. 

Guide  for  Attribute  Lot  Sarrqrling  Inspection  and  MIL-STD-I05,  MIL-HDBK-53-1A, 
1  February  1982,  superseding  MlL-HDBK-53, 30  June  1965. 

Interim  Guidance  on  the  Application  of  ISO  9000! ASQC  Q90  Series,  Quality  System 
Standards,  MIL-HDBK-9000,  NASA-HDBK-9000, 4  February  1994. 

Procurement  Quality  Handbook,  Naval  Surface  Warfare  Onter,  Crane  Division. 

Program  for  Continuous  Process  Improvement,  Guide  for  the  Assessment  and  Certification 
of  I^velopment,  Manufacturing  and  Maintenance  Facilities,  U.S.  Army  Materiel  Cotimiand 
Commitment  to  Excellence,  AMC  Handbook-XXX,  Draft,  12  March  1991. 


Rubber  Manufacturers  Association  (RMA) 

Supplier  Quality  System,  Self-Survey,  RMA  SP-1,  Washington,  DC,  1991. 

Supplier  Quality  System,  On-Site  Survey,  RMA  SP-2,  Washington,  DC,  1991. 

Siqrplier  Quality  System,  Scoring  Guidelines  for  On-Site  Survey,  RMA  SP-3, 
Washington,  DC,  1991. 
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Product  Description  and  Quality  Control  Report  for  Silicon  and  Fluorosilicone  Bases  and 
Compounds,  Technical  Bulletin,  RMA  SP-910.  Washington,  DC,  1991. 


SEMATECH 

Cycle  Time  Improvement  Guidebook,  Technology  Transfer,  No.  92101309A-GEN, 
Austin,  TX,  1992. 

Partnering  for  Total  Quality,  Standardized  Supplier  Quality  Assessment  (SSQA),  prepared 
for  the  U.S.  Semiconductor  Industry,  Austin,  TX,  15  Febraary  1993. 

Partnering  for  Total  Quality,  Volume  1,#90060277B-GEN,  Austin,  TX,  1990. 

Partnering  for  Total  Quality,  A  Partnering  Guidebook,  Volume  4,  No.  90060277B-GEN, 
Austin,  TX,  1990. 

Partnering  for  Total  Quality,  A  Partnering  Guidebook,  Volume  5,  No.  90060278B-GEN, 
Austin,  IX,  1990. 

Partnering  for  Total  Quality,  A  Total  Quality  Tool  Kit,  Volume  6,  No.  90060279A-GEN, 
Austin,  TX,  1991. 

Partnering  for  Total  Quality,  Joint  Assessment  Guide,  Volume  8,  No.  91020461A-GEN, 
Austin,  TX,  1991. 

Partnering  for  Total  Quality,  Executive’s  Implementation  Guide,  Volume  9,  No. 
92041070A-GEN.  Austin,  TX,  1992. 


REPORTS 

DoD/NIST  Workshop  on  Quality  Technology  and  Statistical  Methods  for  D^ense  Systems: 
An  Agenda  for  Improvement,  U.S.  Department  of  Commerce,  National  Institute  of 
Standards  and  Technology,  Gaithersburg,  MD,  10-11  July  1989. 

Center  for  Strategic  and  International  Studies  (CSIS),  Road  Map  for  MILSPEC  Reform, 
Integrating  Commercial  and  Military  Manufacturing,  The  Report  of  the  Working  Group  ai 
Notary  Spe^caticms  and  Standards,  1993. 

Office  of  Technology  Assessment  (OTA),  Global  Standards,  Building  Blocks  for  the 
Future,  Ccmgress  of  the  United  States,  OTA-TCT-512,  March  1^2. 

Report  on  the  DoD  Quality  Leadership  Forum  I,  Defense  System  Management,  Fort 
Belvoir,  VA,  4  October  1991,  with  white  papers. 

Tab 

1 .  Integrated  Product  Development 

2.  Vaiis^ility  Reductitm  Process 

4.  Perfonnance-Based  Management 
6.  In-Plant  (^ality  Evaluation 
8 .  Best  Manufacturing  Practices 

1 0.  ISO  Quality  Assurance  Standards 

11.  (Quality  in  Source  Selection 

1 3.  Systems  Engineering,  Revision  of  MIL-STD-499B 
15.  Transition  from  Development  to  Production 

17.  ^emplary  Facility,  Phases  I  and  n 

1 8.  Qear  Accountability  in  Design 
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20.  Concunent  Engineering 


Report  on  the  DoD  Quality  Leadership  Forum  II,  Summary  Report,  Defense  System 
Management,  Fort  Belvoir,  VA,  15  November  1991. 


JOURNALS 

Aerospace  Industries  Association  (AIA),  Netvsletters,  Volume  4,  Numbers  2-S, 
July-November  1991. 

Institute  of  Electrical  and  Electronics  Engineers  (IEEE),  Inc.,  Standards  Bearer,  Volume  S, 
Number  2,  October  1991. 

American  Society  for  Testing  and  Materials  (ASTM),  Standardization  News,  Electrical  and 
Electronics  Standards,  November  1991. 


CATALOGS 

American  Welding  Society  (AWS),  Publications  Catalog,  Miami,  R,  Fall  1991. 

ASTM,  Standards  Catalog,  Philadelphia,  PA.  Winter  1991. 

AS(JC,  Bringing  Quality  to  Life,  1994  Publications  Catalog,  Summer-Fall,  Milwaukee, 
WI. 

Information  Handling  Services  (IMS),  Standards  Directories. 

Institute  of  Electrical  and  Electronics  Engineers  (IEEE),  Inc.,  Standards  Catalog, 
Piscataway,  NJ,  Fall  1991. 

Institute  for  Interconnecting  and  Packaging  Electronic  Circuits  (IPC),  IPC  Technical 
Publications,  Artwork  and  Test  Boards,  Educational  Videotapes,  Lincolnwood,  IL. 

Global  Engineering  Documents,  1991  Catalog  of  Electronic  Industries  Association  (EIA) 
and  JEDEC  Standards  and  Engineering  Publications,  Washington,  DC. 

Rubber  Manufacturers  Association  (RMA),  Publications  Catalog,  Supplier  Quality 
Assurance,  Washington,  DC,  January  1992. 

Society  of  Automotive  Engineers  (SAE),  1991  SAE  Aerospace  Standards  Index, 
Cooperative  Engineering  Program,  Warrendale,  PA,  February  1991. 

Society  of  Automotive  Engineers  (SAE),  SAE  Aerospace  Material  Specifications  (AMS) 
Index,  Cooperative  Engineering  Program,  Warrendale,  PA,  July  1991. 

Underwriters  Laboratories,  Inc.,  Standards  for  Scfety,  Product  Index,  December  1989. 


BOOKS 

Carr,  David  K.  and  Ian  D.  littman.  Excellence  in  Government:  Total  Quality  Management 
in  the  1990s,  Coopers  and  Lybrand,  1990. 
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Appendix  F 

DEFINITIONS  AND  ACRONYMS 


Quality  means  the  composite  of  material  attributes  including  performance  features  and 
characteristics  of  product  or  service  to  satisfy  a  given  need  (DFAR  246.101). 

Government  Contract  Quality  Assurance  means  the  various  functions,  including  inspection, 
performed  by  the  Government  to  determine  whether  a  contractor  has  fulfilled  the  contract 
obligations  pertaining  to  quality  and  quantity  (FAR  46.101). 


ABCA 

America,  Britain,  Canada, 

IDA 

Institute  for  Defense  Analyses 

Australia 

IPPD 

Integrated  Product  and  Process 

ADPA 

American  Defense  Preparedness 

Devdt^nnent 

Association 

IQUE 

In-plant  (Quality  Evaluation 

AF 

Air  Force 

ISO 

International  (Organization  of 

ANSI 

American  National  Standards 

Standards 

Institute 

LTPD 

Lot  Tolerance  Percent  Defective 

AcZ 

Accept-on-2^  (Defects) 

NASA 

National  Aeronautical  and  Space 

AC]L 

Accq)table  (Quality  Level 

Admiiustration 

ASQC 

American  Socie^  for  Quality 

OSD 

Office  of  the  Secretary  of 

Control 

Defense 

DCMC 

Defense  (jontract  Management 

QA 

(Quality  Assurance 

Command 

SPC 

Statistical  Process  Control 

DFAR 

Defense  Federal  Acquisition 
Regulations 

SQC 

Statistical  (Quality  Ckmtrol 

IXA 

Defense  Logistics  Agency 

TOP 

Technical  Data  Package 

DoD 

Department  of  Drfense 

Total  (Quality  Management 

FAR 

Federal  Acquisition  Regulations 
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